Module index

index.py — Build-time file manifest and a friendly static index.html for Surge (or any static host)

Features - Scans a root directory with an exclusion glob list - Writes a single shared files.json - Writes a per-folder HTML page (root + every subfolder) * Each folder page opens with #folder=/that/folder by default - Mobile-friendly: hamburger toggles the sidebar (overlay on phones) - Density toggle (Comfortable/Compact); row height via CSS vars - Manifest uses href to point to code-viewer HTML (e.g., mycode.py.html) - Code→HTML via Pygments (default on), mtime-aware; –overwrite to force rebuilds - Safer defaults (scripts only), never re-wraps .html/.json/.md/etc. - Embedded JSON (default on) so pages work over file:// (no server) - Resizable table columns with drag handles; widths persist in localStorage - Optionally hide generated artifacts (manifest file and per-folder page) from the listing (default on)

Usage

python index.py –root . [–json files.json] [–html-out files.html] [–exclude PATTERN …] [–title "My Files"] [–pretty | –no-pretty] [–code-html | –no-code-html] [–code-ext .py .sh …] [–code-max-bytes 2621440] [–code-html-out "{relpath}.html"] [–overwrite] [–embed-json | –no-embed-json] [–hide-generated | –no-hide-generated] [–no-hide-code-html | hide_code_html]

Standard options –root (default=".") Root directory to scan. –json (default="files.json") Output JSON path (relative to root). –html-out (default="files.html") Filename for per-folder index pages (root + every folder). –exclude (default=DEFAULT_EXCLUDES) Glob patterns to exclude. –title (default="Directory Index") Title shown in the UI (also in code viewers). –pretty / –no-pretty (default=–pretty) Pretty-print JSON or compact it.

Code→HTML options –code-html (default on) / –no-code-html Enable/disable code-to-HTML rendering with Pygments. –code-ext … (default: scripts only; see DEFAULT_CODE_EXT below) Extensions treated as code (case-insensitive). –code-max-bytes N (default=2621440 bytes) Max source size to render to HTML (skips larger files). –code-html-out PATTERN (default="{relpath}.html") Pattern for code HTML outputs. Placeholders: {relpath}, {dir}, {name}, {stem}, {ext}

Overwrite + embed + listing options –overwrite Force overwrite of generated HTML files (per-folder pages + code viewers). –embed-json (default) / –no-embed-json Embed the manifest JSON into each HTML so pages work via file://. –hide-generated (default) / –no-hide-generated Hide the manifest file and per-folder page itself from the listing. –no-hide-code-html (default) / –hide_code_html Hide generated code-viewer HTML files from the listing (default).

Zip Options

–zip / –no-zip Enable or disable creation of a single ZIP archive containing all discovered files. Default: –zip (enabled).

–zip-name NAME Name of the output ZIP archive (written at root). If set to "auto", the ZIP is named after the scanned root folder (e.g. "./project" → "project.zip"). Default: "auto".

–zip-max-bytes N Skip files larger than N bytes when creating the ZIP. Use 0 for no limit. Default: 0 (no limit).

–zip-include-generated / –no-zip-include-generated Control whether generated control files (the manifest JSON and per-folder HTML pages) are included in the ZIP. Default: –zip-include-generated (include them).

–zip-include-code-html / –no-zip-include-code-html Control whether generated code-viewer HTML files (e.g. "script.py.html") are included in the ZIP. Default: –zip-include-code-html (include them).

–zip-method {deflate,lzma,bzip2} Compression algorithm to use inside the ZIP: deflate → Standard DEFLATE (widely supported, good balance of speed/size). lzma → LZMA/XZ compression (smaller files, slower, less universal). bzip2 → BZIP2 compression (midpoint between deflate and lzma). Default: deflate.

–zip-level N Compression level to use (range 0–9). Applies only to deflate and bzip2. Higher values give smaller files but slower compression. Ignored when –zip-method=lzma. Default: 9 (maximum compression).

Quick examples: # Rename per-folder index pages to files.html and force overwrite

python index.py –html-out files.html –overwrite

# Change code→HTML naming to src/foo.py → src/foo.source.html

python index.py –code-html-out "{dir}/{stem}.source.html"

# No embedding (keep fetch behavior; needs a server)

python index.py –no-embed-json

# Zip management

./index.py –html-out files.html –overwrite –title "FSPB paper - version 3 - files content" –zip –zip-name all-files.zip

Typical example: python index.py –html-out files.html –overwrite –title "FSPB paper - version 3 - files content"

Other examples: python index.py –root . –pretty –title "Directory Index" python index.py –root . –exclude "." ".git" "node_modules" "pycache" "*.map"

Generative Simulation Initiative 🌱 — Dr Olivier Vitrac — contact: olivier.vitrac@gmail.com Revision: 2025-09-24

TIP (for backup): zip -r -9 "$(whoami)@$(hostname)$(basename "$PWD")$(date +'%Y-%m-%d_%H-%M-%S').zip" .

Expand source code
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
index.py — Build-time file manifest and a friendly static index.html for Surge (or any static host)

Features
- Scans a root directory with an exclusion glob list
- Writes a single shared files.json
- Writes a per-folder HTML page (root + every subfolder)
  * Each folder page opens with #folder=/that/folder by default
- Mobile-friendly: hamburger toggles the sidebar (overlay on phones)
- Density toggle (Comfortable/Compact); row height via CSS vars
- Manifest uses href to point to code-viewer HTML (e.g., mycode.py.html)
- Code→HTML via Pygments (default on), mtime-aware; --overwrite to force rebuilds
- Safer defaults (scripts only), never re-wraps .html/.json/.md/etc.
- Embedded JSON (default on) so pages work over file:// (no server)
- Resizable table columns with drag handles; widths persist in localStorage
- Optionally hide generated artifacts (manifest file and per-folder page) from the listing (default on)

Usage:
  python index.py --root . [--json files.json] [--html-out files.html]
                  [--exclude PATTERN ...] [--title "My Files"]
                  [--pretty | --no-pretty]
                  [--code-html | --no-code-html] [--code-ext .py .sh ...]
                  [--code-max-bytes 2621440]
                  [--code-html-out "{relpath}.html"]
                  [--overwrite]
                  [--embed-json | --no-embed-json]
                  [--hide-generated | --no-hide-generated]
                  [--no-hide-code-html | hide_code_html]

Standard options
  --root (default=".")
        Root directory to scan.
  --json (default="files.json")
        Output JSON path (relative to root).
  --html-out (default="files.html")
        Filename for per-folder index pages (root + every folder).
  --exclude (default=DEFAULT_EXCLUDES)
        Glob patterns to exclude.
  --title (default="Directory Index")
        Title shown in the UI (also in code viewers).
  --pretty / --no-pretty (default=--pretty)
        Pretty-print JSON or compact it.

Code→HTML options
  --code-html (default on) / --no-code-html
        Enable/disable code-to-HTML rendering with Pygments.
  --code-ext ... (default: scripts only; see DEFAULT_CODE_EXT below)
        Extensions treated as code (case-insensitive).
  --code-max-bytes N (default=2621440 bytes)
        Max source size to render to HTML (skips larger files).
  --code-html-out PATTERN (default="{relpath}.html")
        Pattern for code HTML outputs. Placeholders: {relpath}, {dir}, {name}, {stem}, {ext}

Overwrite + embed + listing options
  --overwrite
        Force overwrite of generated HTML files (per-folder pages + code viewers).
  --embed-json (default) / --no-embed-json
        Embed the manifest JSON into each HTML so pages work via file://.
  --hide-generated (default) / --no-hide-generated
        Hide the manifest file and per-folder page itself from the listing.
  --no-hide-code-html (default) / --hide_code_html
        Hide generated code-viewer HTML files from the listing (default).

Zip options
-----------
--zip / --no-zip
    Enable or disable creation of a single ZIP archive containing all discovered files.
    Default: --zip (enabled).

--zip-name NAME
    Name of the output ZIP archive (written at root). If set to "auto", the ZIP is
    named after the scanned root folder (e.g. "./project" → "project.zip").
    Default: "auto".

--zip-max-bytes N
    Skip files larger than N bytes when creating the ZIP. Use 0 for no limit.
    Default: 0 (no limit).

--zip-include-generated / --no-zip-include-generated
    Control whether generated control files (the manifest JSON and per-folder HTML pages)
    are included in the ZIP.
    Default: --zip-include-generated (include them).

--zip-include-code-html / --no-zip-include-code-html
    Control whether generated code-viewer HTML files (e.g. "script.py.html") are included
    in the ZIP.
    Default: --zip-include-code-html (include them).

--zip-method {deflate,lzma,bzip2}
    Compression algorithm to use inside the ZIP:
      deflate  → Standard DEFLATE (widely supported, good balance of speed/size).
      lzma     → LZMA/XZ compression (smaller files, slower, less universal).
      bzip2    → BZIP2 compression (midpoint between deflate and lzma).
    Default: deflate.

--zip-level N
    Compression level to use (range 0–9). Applies only to deflate and bzip2.
    Higher values give smaller files but slower compression.
    Ignored when --zip-method=lzma.
    Default: 9 (maximum compression).

Quick examples:
  # Rename per-folder index pages to files.html and force overwrite
  >> python index.py --html-out files.html --overwrite

  # Change code→HTML naming to src/foo.py → src/foo.source.html
  >> python index.py --code-html-out "{dir}/{stem}.source.html"

  # No embedding (keep fetch behavior; needs a server)
  >> python index.py --no-embed-json

  # Zip management
  >> ./index.py --html-out files.html --overwrite \
    --title "FSPB paper - version 3 - files content" \
    --zip --zip-name all-files.zip

Typical example:
  python index.py --html-out files.html --overwrite --title "FSPB paper - version 3 - files content"

Other examples:
  python index.py --root . --pretty --title "Directory Index"
  python index.py --root . --exclude ".*" ".git*" "node_modules" "__pycache__" "*.map"

Generative Simulation Initiative 🌱 — Dr Olivier Vitrac — contact: olivier.vitrac@gmail.com
Revision: 2025-09-24

TIP (for backup):
zip -r -9 "$(whoami)@$(hostname)_$(basename "$PWD")_$(date +'%Y-%m-%d_%H-%M-%S').zip" .

"""

from __future__ import annotations
import argparse
import fnmatch
import json
import mimetypes
import os, sys, zipfile
from datetime import datetime, timezone
from pathlib import Path
from typing import Iterable, Dict, Any, List, Tuple, Set, Optional

# ---------- Defaults ----------
DEFAULT_EXCLUDES = [
    ".*", ".git*", "node_modules", "__pycache__", "*.pyc", "*.pyo",
    "*.map", "dist", "build", ".DS_Store",
]

# SAFE scripts-only default (override with --code-ext if desired)
DEFAULT_CODE_EXT = [
    ".py",".sh",".m",
    ".js",".ts",".jsx",".tsx",
    ".c",".cpp",".h",".hpp",".java",".rs",".go",
    ".rb",".php",".pl",".lua",".r",".sql",
    ".bat",".ps1",
]

DEFAULT_CODE_MAX_BYTES = 2_621_440  # ~2.5 MB

try:
    from pygments import highlight
    from pygments.formatters import HtmlFormatter
    from pygments.lexers import get_lexer_for_filename, TextLexer
    HAVE_PYGMENTS = True
except Exception:
    HAVE_PYGMENTS = False

# ---------- UI HTML Template ----------
HTML_TEMPLATE = r"""<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>{{TITLE}}</title>
<meta name="description" content="Static manifest index">
<style>
  :root{
    --bg:#0b0f14; --fg:#e7edf4; --muted:#a8b3c0; --accent:#59d98a; --accent-dim:#2e7d5b;
    --band:#0f141b; --card:#0d1218; --border:#17202a; --link:#9bd3ff; --row-hover:#111923;
    --mono: ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;
    --sans: Inter, system-ui, -apple-system, Segoe UI, Roboto, Ubuntu, Cantarell, Noto Sans, Arial, sans-serif;
    --font-size: 14px; --pad-y: 8px; --pad-x: 10px; --header-pad-y: 8px; --header-pad-x: 10px; --sidebar-w: 280px;
  }
  body.compact{ --font-size: 12px; --pad-y: 2px; --pad-x: 3px; --header-pad-y: 2px; --header-pad-x: 3px; }
  *{box-sizing:border-box} html,body{height:100%}
  body{margin:0; background:var(--bg); color:var(--fg); font-family:var(--sans); display:flex; min-height:100vh; font-size:var(--font-size)}
  a{color:var(--link); text-decoration:none} a:hover{text-decoration:underline}
  .app{display:grid; grid-template-columns: var(--sidebar-w) 1fr; grid-template-rows:auto 1fr auto; width:100%}
  header{grid-column:1/-1; padding:12px 14px; border-bottom:1px solid var(--border); display:flex; align-items:center; gap:12px; flex-wrap:wrap; background:var(--card)}
  .hamburger{display:inline-flex; align-items:center; justify-content:center; width:36px; height:36px; border:1px solid var(--border); background:var(--band); color:var(--fg); border-radius:8px; cursor:pointer}
  .hamburger:hover{background:#131c25}
  .brand{font-weight:700; letter-spacing:.2px; display:flex; align-items:center; gap:10px}
  .brand .seed{font-size:20px}
  .muted{color:var(--muted)}
  .spacer{flex:1 1 auto}
  .stats{display:flex; gap:10px; flex-wrap:wrap}
  .chip{background:var(--band); border:1px solid var(--border); padding:4px 8px; border-radius:999px; font-size:12px}
  .sidebar{grid-row:2/span 1; border-right:1px solid var(--border); padding:12px; overflow:auto; background:linear-gradient(180deg, #0c1218, #0a0f14)}
  .content{grid-row:2/span 1; display:flex; flex-direction:column; min-width:0}
  .toolbar{padding:8px 12px; border-bottom:1px solid var(--border); display:flex; gap:10px; flex-wrap:wrap; align-items:center; background:var(--card)}
  .search{flex:1 1 280px; display:flex; gap:8px; align-items:center; background:var(--band); border:1px solid var(--border); border-radius:10px; padding:6px 8px}
  .search input{flex:1; background:transparent; border:0; outline:0; color:var(--fg); font-family:var(--sans)}
  .crumbs{display:flex; gap:6px; align-items:center; flex-wrap:wrap; font-size:14px}
  .crumbs .seg{padding:2px 6px; background:var(--band); border:1px solid var(--border); border-radius:8px}
  .btn{display:inline-flex; align-items:center; gap:6px; padding:6px 8px; border:1px solid var(--border); background:var(--band); color:var(--fg); border-radius:10px; cursor:pointer; text-decoration:none; font-size:13px}
  .btn:hover{background:#131c25}
  .table-wrap{overflow:auto; flex:1}
  table{width:100%; border-collapse:separate; border-spacing:0}
  /* Column widths */
  #grid th[data-sort="name"]  { width: 40%; }
  #grid th[data-sort="ext"]   { min-width: 60px; }
  #grid th[data-sort="mime"]  { min-width: 120px; }
  #grid th[data-sort="size"]  { min-width: 80px; text-align:right; }
  #grid th[data-sort="mtime_epoch"] { min-width: 160px; }
  #grid th:last-child         { width: 5%; }
  thead th{position:sticky; top:0; background:var(--card); z-index:1; text-align:left; padding:var(--header-pad-y) var(--header-pad-x); border-bottom:1px solid var(--border); white-space:nowrap; cursor:pointer; user-select:none}
  tbody td{padding:var(--pad-y) var(--pad-x); border-bottom:1px solid var(--border); vertical-align:middle}
  tbody tr:nth-child(even){background:var(--band)} tbody tr:hover{background:var(--row-hover)}
  .name{display:flex; align-items:center; gap:8px}
  .icon{width:18px; height:18px; display:inline-flex; align-items:center; justify-content:center; font-size:16px}
  .mono{font-family:var(--mono)} .sort-caret{opacity:.6; margin-left:6px} .copy-btn{opacity:.8} .copy-btn:hover{opacity:1}
  /* Right-align numeric/date cells */
  #grid td:nth-child(4), #grid td:nth-child(5) { text-align:right; }
  /* Truncate long names */
  #grid td .name a { display:inline-block; max-width: clamp(220px, 40vw, 900px); overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
  .footer{grid-column:1/-1; border-top:1px solid var(--border); padding:10px 12px; display:flex; gap:10px; align-items:center; justify-content:space-between; background:var(--card); font-size:13px}
  .tree{list-style:none; margin:0; padding:0} .tree li{margin:2px 0} .tree .folder > a{font-weight:600}
  .tree a{display:inline-block; padding:3px 6px; border-radius:6px} .tree a.active{background:var(--accent-dim); color:#fff}
  /* Column resizer handle */
  th { position: relative; }
  .resizer { position:absolute; right:0; top:0; width:6px; height:100%; cursor:col-resize; user-select:none; }
  .resizer:hover { background: rgba(255,255,255,.06); }
  /* Preview lightbox */
  .preview-backdrop {
    position: fixed; inset: 0; background: rgba(0,0,0,.7);
    display: none; align-items: center; justify-content: center;
    z-index: 1000;
  }
  .preview-backdrop.open { display: flex; }
  .preview-modal {
    background: var(--card);
    border: 1px solid var(--border);
    border-radius: 12px;
    max-width: min(92vw, 1100px);
    max-height: 88vh;
    width: min(92vw, 1100px);
    overflow: hidden;
    box-shadow: 0 10px 40px rgba(0,0,0,.45);
    display: flex; flex-direction: column;
  }
  .preview-head {
    display: flex; align-items: center; justify-content: space-between;
    padding: 8px 12px; border-bottom: 1px solid var(--border);
  }
  .preview-body {
    background: var(--bg);
    padding: 10px; overflow: auto; display: flex; align-items: center; justify-content: center;
  }
  .preview-foot { padding: 8px 12px; border-top: 1px solid var(--border); display: flex; justify-content: flex-end; gap: 8px; }
  .preview-body img, .preview-body video, .preview-body audio, .preview-body iframe { max-width: 100%; max-height: 76vh; background: #000; }
  /* PDF mode: use (almost) the full viewport */
  .preview-backdrop.pdf .preview-modal {
    max-width: 98vw;
    max-height: 98vh;
    width: 98vw;               /* edge-to-edge on desktop */
  }
  /* give the PDF iframe most of the vertical space */
  .preview-backdrop.pdf .preview-body iframe {
    width: 100%;
    height: calc(98vh - 112px); /* header( ~48px ) + footer( ~48px ) + paddings ≈ 112px */
    border: 0;
  }
  @media (max-width: 900px){
    .app{grid-template-columns: 1fr}
    .sidebar{position: fixed; inset: 0 auto 0 0; width: min(84vw, 320px); transform: translateX(-100%); 
   .preview-backdrop.pdf .preview-modal {
    max-width: 100vw;
    max-height: 100vh;
    width: 100vw;
    border-radius: 0; /* edge-to-edge on mobile */
  }
  .preview-backdrop.pdf .preview-body iframe {
    height: calc(100vh - 112px);
  }
    transition: transform .2s ease; z-index: 20; box-shadow: 8px 0 24px rgba(0,0,0,.35)}
    body.menu-open .sidebar{ transform: translateX(0) }
    body.menu-open::before{content:""; position: fixed; inset:0; background: rgba(0,0,0,.45); z-index: 15}
  }
</style>
</head>
<body>
<div class="app">
  <header>
    <button class="hamburger" id="hamburger" aria-label="Toggle menu" title="Toggle menu">
      <svg width="20" height="20" viewBox="0 0 24 24" aria-hidden="true"><path fill="currentColor" d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
    </button>
    <div class="brand">
      <span class="seed">🌱</span>
      <span>{{TITLE}}</span>
      <span class="muted">· Generative Simulation Initiative — Dr&nbsp;Olivier&nbsp;Vitrac — <a href="mailto:olivier.vitrac@gmail.com">olivier.vitrac@gmail.com</a></span>
    </div>
    <div class="spacer"></div>
    <div class="stats">
      <span class="chip" id="stat-files">0 files</span>
      <span class="chip" id="stat-folders">0 folders</span>
      <span class="chip" id="stat-size">0</span>
      <span class="chip" id="stat-generated">generated —</span>
    </div>
    <button class="btn" id="density">Density: Comfortable</button>
  </header>

  <nav class="sidebar" aria-label="Folders">
    <div class="muted" style="margin:4px 6px 10px">Folders</div>
    <ul class="tree" id="tree"></ul>
  </nav>

  <main class="content">
    <div class="toolbar">
      <div class="crumbs" id="crumbs"></div>
      <div class="search">
        <svg width="16" height="16" viewBox="0 0 24 24" aria-hidden="true"><path fill="currentColor" d="M10 2a8 8 0 105.293 14.293l4.207 4.207l1.414-1.414l-4.207-4.207A8 8 0 0010 2m0 2a6 6 0 110 12A6 6 0 0110 4z"/></svg>
        <input id="q" placeholder="Search names, extensions, mime…" autocomplete="off">
        <button class="btn" id="clear">Clear</button>
      </div>
      <a class="btn" id="toggle-folders">Show files in subfolders</a>
    </div>
    <div class="table-wrap">
      <table id="grid">
        <thead>
          <tr>
            <th data-sort="name">Name <span class="sort-caret">↕</span></th>
            <th data-sort="ext">Ext <span class="sort-caret">↕</span></th>
            <th data-sort="mime">Type <span class="sort-caret">↕</span></th>
            <th data-sort="size">Size <span class="sort-caret">↕</span></th>
            <th data-sort="mtime_epoch">Modified <span class="sort-caret">↕</span></th>
            <th>Link</th>
          </tr>
        </thead>
        <tbody id="rows"></tbody>
      </table>
    </div>
  </main>

  <footer class="footer">
    <div><span class="muted">Index powered by a static manifest.</span></div>
    <div>
      <a class="btn" href="files.json">View manifest</a>
      <a class="btn" href="#" id="download-all" style="display:none">Download all (zip)</a>
      <a class="btn" href="#" id="copy-current">Copy folder URL</a>
    </div>
  </footer>
</div>

<!-- Preview lightbox -->
<div class="preview-backdrop" id="preview">
  <div class="preview-modal">
    <div class="preview-head">
      <div class="mono" id="preview-title">Preview</div>
      <div>
        <button class="btn" id="preview-open">Open</button>
        <button class="btn" id="preview-close">Close</button>
      </div>
    </div>
    <div class="preview-body" id="preview-body"></div>
    <div class="preview-foot"></div>
  </div>
</div>

<!-- If --embed-json is used, the manifest will be injected right here -->
{{EMBED_MANIFEST}}

<script>
const EMBED = document.getElementById("manifest");
const MANIFEST_URL = "files.json";
const rowsEl = document.getElementById("rows");
const treeEl = document.getElementById("tree");
const crumbsEl = document.getElementById("crumbs");
const qEl = document.getElementById("q");
const clearEl = document.getElementById("clear");
const toggleEl = document.getElementById("toggle-folders");
const statFiles = document.getElementById("stat-files");
const statFolders = document.getElementById("stat-folders");
const statSize = document.getElementById("stat-size");
const statGen = document.getElementById("stat-generated");
const copyCur = document.getElementById("copy-current");
const hamburger = document.getElementById("hamburger");
const densityBtn = document.getElementById("density");
const downloadAll = document.getElementById("download-all");

let manifest = null;
let currentFolder = "/";
let showDeep = true;
let sortKey = "name";
let sortDir = 1;
let q = "";
let density = "comfortable";

/* utils */
function humanSize(n){const u=["B","KB","MB","GB","TB"];let i=0,x=n;while(x>=1024&&i<u.length-1){x/=1024;i++;}return(i===0?x:x.toFixed(1))+" "+u[i]}
function fmtDate(e){const d=new Date(e*1000);return d.toISOString().replace("T"," ").replace(".000Z"," UTC")}
function iconFor(ext,mime){
  ext=(ext||"").toLowerCase();
  if (ext === ".py") return "🐍";
  if (ext === ".m") return "Ⓜ️";
  if (ext === ".md") return "📑";
  if (ext === ".sh") return "🐚";
  if (ext === ".svg") return "📈";
  if (ext === ".tex") return "🔤";
  if (ext === ".bib") return "🍼";
  if (ext === ".pdf") return "📕";
  if (ext === ".csv") return "📊";
  if (ext === ".png") return "👁️";
  if (ext === ".gif") return "🎥";
  if([".htm",".html"].includes(ext)) return "🌐";
  if([".jpg",".jpeg"].includes(ext)) return "📷";
  if(mime&&mime.startsWith("image/")) return "🖼️";
  if(mime&&mime.startsWith("audio/")) return "🎵";
  if(mime&&mime.startsWith("video/")) return "🎬";
  if([".doc",".docx",".odt",".rtf"].includes(ext)) return "📘";
  if([".xls",".xlsx",".ods",".csv","/excel"].some(x=>ext===x||(mime||"").includes(x))) return "📗";
  if([".ppt",".pptx",".odp"].includes(ext)) return "🖥️";
  if([".zip",".tar",".gz",".bz2",".7z",".rar"].includes(ext)) return "🗜️";
  if([".py",".ipynb",".js",".ts",".jsx",".tsx",".html",".css",".json",".yml",".yaml",".toml",".ini",
      ".sh",".bat",".ps1",".m",".cpp",".c",".h",".hpp",".rs",".go",".java"].includes(ext)) return "🧩";
  return "📄";
}

/* sidebar tree */
function updateTree(){
  treeEl.innerHTML=""; const folders=new Set(["/"]);
  for(const f of manifest.files){const parts=f.path.split("/").filter(Boolean);let acc="";for(let i=0;i<parts.length-1;i++){acc+="/"+parts[i];folders.add(acc||"/");}}
  const tree={}; for(const fol of folders){if(fol==="/")continue; const parts=fol.split("/").filter(Boolean); let cur=tree; for(const seg of parts){cur[seg]=cur[seg]||{}; cur=cur[seg];}}
  treeEl.appendChild(walkTree(tree,""));
}
function walkTree(node,base=""){const keys=Object.keys(node).sort((a,b)=>a.localeCompare(b)); const ul=document.createElement("ul"); ul.className="tree";
  for(const k of keys){const li=document.createElement("li"); li.className="folder"; const a=document.createElement("a"); const full=(base+"/"+k).replace(/\/+/g,"/"); a.href="#folder="+encodeURIComponent(full); a.textContent="📁 "+k; if(full===currentFolder)a.classList.add("active"); li.appendChild(a); li.appendChild(walkTree(node[k],full)); ul.appendChild(li);} return ul;}

/* crumbs/filter/sort */
function renderCrumbs(){crumbsEl.innerHTML=""; const base=document.createElement("a"); base.href="#folder="+encodeURIComponent("/"); base.textContent="root"; base.className="seg"; crumbsEl.appendChild(base); const parts=currentFolder.split("/").filter(Boolean); let acc=""; for(const seg of parts){acc+="/"+seg; const a=document.createElement("a"); a.href="#folder="+encodeURIComponent(acc); a.textContent=seg; a.className="seg"; crumbsEl.appendChild(a);}}
function applyFilter(){const qx=q.trim().toLowerCase(); return manifest.files.filter(f=>{if(currentFolder!=="/"){if(showDeep){if(!f.path.startsWith(currentFolder.slice(1)+"/")&&!f.path.startsWith(currentFolder.slice(1)+"\\"))return false;}else{const dir=f.path.split("/").slice(0,-1).join("/"); if((currentFolder.slice(1)||"")!==dir)return false;}} if(!qx)return true; const hay=(f.name+" "+(f.ext||"")+" "+(f.mime||"")).toLowerCase(); return hay.includes(qx);});}
function sortFiles(list){const key=sortKey,dir=sortDir,coll=new Intl.Collator(undefined,{numeric:true,sensitivity:"base"}); list.sort((a,b)=>{let va=a[key],vb=b[key]; if(key==="size"||key==="mtime_epoch"){va=+va;vb=+vb;return(va-vb)*dir;} return coll.compare(String(va||""),String(vb||""))*dir;}); return list;}

/* preview helpers */
function canPreview(mime, ext){
  mime=(mime||"").toLowerCase(); ext=(ext||"").toLowerCase();
  if(mime.startsWith("image/")) return "image";
  if(mime.startsWith("video/")) return "video";
  if(mime.startsWith("audio/")) return "audio";
  if(mime==="application/pdf" || ext===".pdf") return "pdf";
  return null;
}
function openPreview(file){
  const kind = canPreview(file.mime, file.ext);
  if(!kind) return false;
  const bg   = document.getElementById("preview");
  const body = document.getElementById("preview-body");
  const title= document.getElementById("preview-title");
  const openBtn = document.getElementById("preview-open");
  body.innerHTML = "";
  title.textContent = file.name;
  // set/unset PDF mode
  if (kind === "pdf") bg.classList.add("pdf"); else bg.classList.remove("pdf");
  const url = new URL(file.href||file.path, window.location.href).href;
  openBtn.onclick = ()=>{ window.location.href = url; };
  let el = null;
  if (kind === "image"){
    el = document.createElement("img"); el.src = url; el.alt = file.name;
  } else if (kind === "video"){
    el = document.createElement("video"); el.src = url; el.controls = true;
  } else if (kind === "audio"){
    el = document.createElement("audio"); el.src = url; el.controls = true; el.style.width = "100%";
  } else if (kind === "pdf"){
    el = document.createElement("iframe"); el.src = url; el.setAttribute("title", file.name);
  }
  if (el){ body.appendChild(el); bg.classList.add("open"); }
  return true;
}
function attachPreviewHandlers(){
  rowsEl.querySelectorAll("tr").forEach((tr)=>{
    const a = tr.querySelector("td.name a"); if(!a) return;
    const f = a._fileMeta; if(!f) return;
    if(!canPreview(f.mime, f.ext)) return;
    a.addEventListener("click",(e)=>{ e.preventDefault(); openPreview(f); });
  });
  const bg=document.getElementById("preview");
  const closeBtn=document.getElementById("preview-close");
  const onClose = ()=> { bg.classList.remove("open"); bg.classList.remove("pdf"); };
  bg.addEventListener("click",(e)=>{ if(e.target===bg) onClose(); });
  closeBtn.addEventListener("click", onClose);
  document.addEventListener("keydown",(e)=>{ if(e.key==="Escape") onClose(); });
}

/* table */
function renderTable(){
  const filtered = sortFiles(applyFilter());
  rowsEl.innerHTML = "";
  for(const f of filtered){
    const tr = document.createElement("tr");

    const tdName = document.createElement("td"); tdName.className="name";
    const icon = document.createElement("span"); icon.className="icon"; icon.textContent = iconFor(f.ext, f.mime);
    const link = document.createElement("a"); link.href = encodeURI(f.href || f.path); link.textContent = f.name; link.className="mono";
    link._fileMeta = f; // for preview
    tdName.appendChild(icon); tdName.appendChild(link);

    const tdExt = document.createElement("td"); tdExt.textContent = f.ext||""; tdExt.className="mono";
    const tdMime = document.createElement("td"); tdMime.textContent = f.mime||""; tdMime.className="mono";
    const tdSize = document.createElement("td"); tdSize.textContent = humanSize(f.size); tdSize.className="mono";
    const tdMtime = document.createElement("td"); tdMtime.textContent = fmtDate(f.mtime_epoch); tdMtime.className="mono";

    const tdLink = document.createElement("td");
    const btn = document.createElement("a"); btn.href="#"; btn.className="btn copy-btn"; btn.textContent="Copy";
    btn.addEventListener("click",(e)=>{
      e.preventDefault();
      const url = new URL(f.href || f.path, window.location.href).href;
      navigator.clipboard.writeText(url).then(()=>{ btn.textContent="Copied!"; setTimeout(()=>btn.textContent="Copy", 1200); });
    });
    tdLink.appendChild(btn);

    tr.appendChild(tdName); tr.appendChild(tdExt); tr.appendChild(tdMime); tr.appendChild(tdSize); tr.appendChild(tdMtime); tr.appendChild(tdLink);
    rowsEl.appendChild(tr);
  }
  attachPreviewHandlers();
}

/* sort handlers */
function attachSortHandlers(){document.querySelectorAll("thead th[data-sort]").forEach(th=>{th.addEventListener("click",()=>{const k=th.getAttribute("data-sort"); if(sortKey===k){sortDir*=-1;}else{sortKey=k;sortDir=1;} renderTable();});});}

/* stats */
function updateStats(){statFiles.textContent=`${manifest.summary.files} files`; statFolders.textContent=`${manifest.summary.folders} folders`; statSize.textContent=humanSize(manifest.summary.total_size); statGen.textContent=`generated — ${manifest.generated_at}`}

/* zip button */
function updateZipButton(){
  if (!downloadAll || !manifest || !manifest.summary) return;
  const s = manifest.summary;
  if (s.zip_exists && s.zip_name){
    downloadAll.href = s.zip_name;
    const txt = "Download all (" + humanSize(s.zip_compressed) + " / " +
                humanSize(s.zip_uncompressed) + ")";
    downloadAll.textContent = txt;
    downloadAll.title = "Compressed: " + humanSize(s.zip_compressed) +
                        " — Uncompressed: " + humanSize(s.zip_uncompressed);
    downloadAll.style.display = "inline-flex";
  } else {
    downloadAll.style.display = "none";
  }
}


/* URL state */
function readHash(){const h=new URLSearchParams(location.hash.slice(1)); return {folder:decodeURIComponent(h.get("folder")||"/"), q:h.get("q")||"", deep:h.get("deep")!=="0", density:(h.get("density")||"comfortable")};}
function writeHash(){const p=new URLSearchParams(); p.set("folder",currentFolder); if(q)p.set("q",q); p.set("deep",showDeep?"1":"0"); p.set("density",density); location.hash=p.toString();}

/* density */
function applyDensity(){ if(density==="compact"){document.body.classList.add("compact"); densityBtn.textContent="Density: Compact";} else {document.body.classList.remove("compact"); densityBtn.textContent="Density: Comfortable";}}

/* manifest loader (embedded first, then fetch) */
async function loadManifest(){
  if(EMBED){ try{ return JSON.parse(EMBED.textContent); }catch(e){ console.error("Embedded manifest parse error", e); } }
  const res = await fetch(MANIFEST_URL); return await res.json();
}

/* column resize (with per-folder persistence) */
function getColWidthKey(){
  const st = readHash();
  return "colwidths:" + location.pathname + "::" + (st.folder || "/");
}
function applySavedColumnWidths(){
  const table = document.getElementById("grid");
  const headers = table.querySelectorAll("th");
  const key = getColWidthKey();
  try{
    const saved = JSON.parse(localStorage.getItem(key) || "null");
    if(saved && Array.isArray(saved)){
      saved.forEach((w,i)=>{ if(w){ headers[i].style.width = w; }});
    }
  }catch(_){}
}
function initResizableColumns(){
  const table = document.getElementById("grid");
  const headers = table.querySelectorAll("th");
  applySavedColumnWidths();
  headers.forEach((th)=>{
    if (th.querySelector(".resizer")) return;
    const resizer = document.createElement("div");
    resizer.className = "resizer";
    th.appendChild(resizer);
    let startX=0, startW=0;
    const onMove = (e)=>{
      const dx = e.clientX - startX;
      const newW = Math.max(40, startW + dx);
      th.style.width = newW + "px";
    };
    const onUp = ()=>{
      document.removeEventListener("mousemove", onMove);
      document.removeEventListener("mouseup", onUp);
      const key = getColWidthKey();
      const widths = [...headers].map(h=>h.style.width || null);
      try{ localStorage.setItem(key, JSON.stringify(widths)); }catch(_){}
    };
    resizer.addEventListener("mousedown", (e)=>{
      startX = e.clientX; startW = th.getBoundingClientRect().width;
      document.addEventListener("mousemove", onMove);
      document.addEventListener("mouseup", onUp);
    });
  });
  window.addEventListener("hashchange", applySavedColumnWidths);
}

/* main */
async function main(){
  manifest = await loadManifest();
  updateStats(); updateZipButton(); attachSortHandlers();

  window.addEventListener("hashchange", ()=>{
    const st=readHash(); currentFolder=st.folder; q=st.q; showDeep=st.deep; density=st.density; qEl.value=q; applyDensity(); renderCrumbs(); updateTree(); renderTable(); applySavedColumnWidths(); updateZipButton();
  });

  const st0=readHash(); currentFolder=st0.folder; q=st0.q; showDeep=st0.deep; density=st0.density; qEl.value=q; applyDensity(); renderCrumbs(); updateTree(); renderTable();

  qEl.addEventListener("input", ()=>{ q=qEl.value; writeHash(); });
  clearEl.addEventListener("click", ()=>{ q=""; qEl.value=""; writeHash(); });
  toggleEl.addEventListener("click",(e)=>{ e.preventDefault(); showDeep=!showDeep; writeHash(); });
  copyCur.addEventListener("click",(e)=>{ e.preventDefault(); const url=new URL(location.href); url.hash=`folder=${encodeURIComponent(currentFolder)}&q=${encodeURIComponent(q)}&deep=${showDeep?1:0}&density=${density}`; navigator.clipboard.writeText(url.href); });
  hamburger.addEventListener("click", ()=>{ document.body.classList.toggle("menu-open"); });
  document.addEventListener("click",(e)=>{ if(document.body.classList.contains("menu-open")){ const sb=document.querySelector(".sidebar"); if(!sb.contains(e.target) && e.target!==hamburger && !hamburger.contains(e.target)){ document.body.classList.remove("menu-open"); } }});
  densityBtn.addEventListener("click", ()=>{ density = (density==="comfortable" ? "compact" : "comfortable"); writeHash(); });

  initResizableColumns();
}
main();
</script>
</body>
</html>
"""

# ---------- Helpers ----------
def match_any(patterns: Iterable[str], name: str) -> bool:
    for p in patterns:
        if fnmatch.fnmatch(name, p):
            return True
    return False

def should_skip(path: Path, excludes: List[str], root: Path) -> bool:
    rel = path.relative_to(root).as_posix()
    parts = rel.split("/")
    if match_any(excludes, rel): return True
    if match_any(excludes, path.name): return True
    for seg in parts:
        if match_any(excludes, seg): return True
    return False

def html_escape(s: str) -> str:
    return s.replace("&","&amp;").replace("<","&lt;").replace(">","&gt;")

def highlight_code_to_html(src_text: str, filename: str, title: str) -> str:
    """Render code to standalone dark-themed HTML with inline CSS (works on file://)."""
    def esc(x: str) -> str:
        return x.replace("&","&amp;").replace("<","&lt;").replace(">","&gt;")
    header = (
        "<div style='padding:8px 10px;"
        "background:#0d1218;border-bottom:1px solid #17202a;color:#e7edf4;"
        "font-family:system-ui,Arial,sans-serif;font-size:14px;'>"
        f"<strong>🌱 {esc(title)}</strong> — viewing <code>{esc(filename)}</code></div>"
    )
    base_css = """
      body{margin:0;background:#0b0f14;color:#e7edf4}
      .highlight{background:#0b0f14 !important; color:#e7edf4 !important;}
      pre{margin:0}
      .highlighttable{border-collapse:separate;border-spacing:0;width:100%}
      .highlighttable td{vertical-align:top}
      .linenos{background:#0b0f14;color:#8aa1b4 !important; padding:0 .5em; user-select:none}
      .linenos pre{background:transparent !important; color:inherit !important}
      .code{background:#0b0f14}
    """
    if not HAVE_PYGMENTS:
        body = f"<pre class='highlight' style='padding:12px;overflow:auto'>{esc(src_text)}</pre>"
        return f"<!doctype html><meta charset='utf-8'><title>{esc(title)} · {esc(filename)}</title><style>{base_css}</style>{header}{body}"
    try:
        try:
            lexer = get_lexer_for_filename(filename, stripall=False)
        except Exception:
            lexer = TextLexer(stripall=False)
        formatter = HtmlFormatter(full=False, noclasses=False, linenos=True)
        pyg_css = formatter.get_style_defs('.highlight')
        highlighted = highlight(src_text, lexer, formatter)
        style = f"<style>{base_css}\n{pyg_css}</style>"
        return f"<!doctype html><meta charset='utf-8'><title>{esc(title)} · {esc(filename)}</title>{style}{header}<div style='padding:0 8px 8px'>{highlighted}</div>"
    except Exception:
        body = f"<pre class='highlight' style='padding:12px;overflow:auto'>{esc(src_text)}</pre>"
        return f"<!doctype html><meta charset='utf-8'><title>{esc(title)} · {esc(filename)}</title><style>{base_css}</style>{header}{body}"

def format_code_out(relpath: str, pattern: str) -> str:
    p = Path(relpath)
    mapping = {
        "relpath": relpath,
        "dir": p.parent.as_posix(),
        "name": p.name,
        "stem": p.stem,
        "ext": p.suffix,
    }
    return pattern.format(**mapping).lstrip("./")

def ensure_code_html(src_path: Path, html_path: Path, title: str, max_bytes: int, overwrite: bool) -> bool:
    try:
        if src_path.stat().st_size > max_bytes:
            return False
        dst_exists = html_path.exists()
        src_m = src_path.stat().st_mtime
        dst_m = html_path.stat().st_mtime if dst_exists else -1
        need = overwrite or (not dst_exists) or (dst_m <= src_m)   # <= fixes same-second ties
        if need:
            text = src_path.read_text(encoding="utf-8", errors="replace")
            html = highlight_code_to_html(text, src_path.name, title)
            html_path.parent.mkdir(parents=True, exist_ok=True)
            html_path.write_text(html, encoding="utf-8")
            return True
        return False
    except Exception:
        return False
    
def _resolve_zip_options(method_str: str, level: int):
    """
    Map CLI method string to zipfile constants and normalize level.
    Returns (compression_const, compresslevel_or_None).
    compresslevel is used only for DEFLATE/BZIP2; LZMA ignores it.
    """
    m = (method_str or "deflate").lower()
    if m == "deflate":
        # zlib levels 0–9 (0 = store, 9 = max)
        lvl = max(0, min(int(level), 9))
        return zipfile.ZIP_DEFLATED, lvl
    if m == "bzip2":
        # bz2 levels 1–9 (python accepts 1–9; clamp)
        lvl = max(1, min(int(level), 9))
        return zipfile.ZIP_BZIP2, lvl
    if m == "lzma":
        # Python’s ZipFile ignores compresslevel for ZIP_LZMA
        return zipfile.ZIP_LZMA, None
    raise ValueError(f"Unknown --zip-method: {method_str!r}")

# ---------- Scanner ----------
def scan(root: Path,
         excludes: List[str],
         code_html: bool,
         code_exts: Set[str],
         max_code_bytes: int,
         site_title: str,
         code_out_pattern: str,
         overwrite: bool,
         json_name: str,
         html_out_name: str,
         hide_generated: bool,
         hide_code_html: bool
         ) -> Tuple[List[Dict[str,Any]], Set[str], int, Set[str], Set[str]]:
    """
    Scan files, optionally emit code HTML, return (files, folders, n_code_html_written).
    Returns: files (for manifest table), folders, n_code_html_written,
          code_targets (all viewer paths), found_paths (all discovered files)
    """
    files: List[Dict[str,Any]] = []
    folders: Set[str] = set(["/"])
    made = 0
    code_targets: Set[str] = set()
    found_paths: Set[str] = set()

    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if not should_skip(Path(dirpath)/d, excludes, root)]

        rel_dir = Path(dirpath).relative_to(root).as_posix()
        if rel_dir != ".":
            acc = ""
            for seg in rel_dir.split("/"):
                acc += "/" + seg
                folders.add(acc)

        for fn in filenames:
            p = Path(dirpath) / fn
            if should_skip(p, excludes, root):
                continue

            try:
                st = p.stat()
            except OSError:
                continue

            size = int(st.st_size)
            mtime_epoch = int(st.st_mtime)
            rel_path = p.relative_to(root).as_posix()
            name = Path(rel_path).name
            ext = Path(fn).suffix
            mime = mimetypes.guess_type(fn, strict=False)[0] or ""

            # record each file (for zip)
            found_paths.add(rel_path)

            # Never convert HTML-like or our own generated control files
            is_htmlish = rel_path.lower().endswith(".html")
            is_manifest = (name == json_name)
            is_index_like = (name == html_out_name)

            # Optionally hide generated artifacts from the listing
            if hide_generated and (is_manifest or is_index_like):
                pass_hidden = True
            else:
                pass_hidden = False

            href: Optional[str] = None

            # Conversion decision
            if code_html:
                if (ext.lower() in code_exts) and (not is_htmlish) and (not is_manifest) and (not is_index_like):
                    out_rel = format_code_out(rel_path, code_out_pattern)
                    # Avoid self-mapping and infinite chains
                    if out_rel != rel_path and not out_rel.lower().endswith(".html.html"):
                        code_targets.add(out_rel)
                        out_abs = (root / out_rel)
                        if ensure_code_html(p, out_abs, site_title, max_code_bytes, overwrite):
                            made += 1
                        href = out_rel

            file_entry = {
                "path": rel_path,               # original path
                "href": href or rel_path,       # target to open
                "name": fn,
                "ext": ext if ext else "",
                "size": size,
                "mtime_iso": datetime.fromtimestamp(mtime_epoch, tz=timezone.utc).isoformat(),
                "mtime_epoch": mtime_epoch,
                "mime": mime,
            }

            if not pass_hidden:
                files.append(file_entry)

    if hide_code_html and code_targets:
      files = [f for f in files if f["path"] not in code_targets]
    return files, folders, made, code_targets, found_paths

# ---------- Writers ----------
def write_json(path: Path, payload: Dict[str,Any], pretty: bool):
    if pretty:
        path.write_text(json.dumps(payload, indent=2, ensure_ascii=False), encoding="utf-8")
    else:
        path.write_text(json.dumps(payload, separators=(",",":"), ensure_ascii=False), encoding="utf-8")

def build_html(title: str, default_folder: str = "/", embed_json: bool = True, manifest: Optional[Dict[str,Any]] = None) -> str:
    """Build the HTML; optionally embed the manifest JSON."""
    html = HTML_TEMPLATE.replace("{{TITLE}}", title)
    if embed_json and manifest is not None:
        embed = "<script id=\"manifest\" type=\"application/json\">" + json.dumps(manifest, ensure_ascii=False) + "</script>"
    else:
        embed = ""
    html = html.replace("{{EMBED_MANIFEST}}", embed)
    injection = f"""
<script>
  if (!location.hash) {{
      location.hash = "#folder={default_folder}";
  }}
</script>
"""
    return html.replace("</body>", injection + "\n</body>")

# ---------- Zip engine ----------
def build_zip(root: Path,
              found_paths: Set[str],
              zip_name: str,
              include_generated: bool,
              include_code_html: bool,
              code_targets: Set[str],
              control_names: Set[str],
              max_bytes: int,
              zip_method: str,  # "deflate" | "lzma" | "bzip2"
              zip_level: int,   # e.g. 9 (DEFLATE/BZIP2 only)
              ) -> Tuple[bool, int, int]:
    """
    Create a zip at root/zip_name containing files from found_paths.
    - Skips the zip itself if already present.
    - If include_generated is False: skip manifest/per-folder page names.
    - If include_code_html is False: skip any path in code_targets.
    - If max_bytes > 0: skip files larger than that threshold.
    Returns (exists, total_bytes_written).
    """
    comp_const, comp_level = _resolve_zip_options(zip_method, zip_level)
    zpath = root / zip_name
    # Avoid zipping the previous zip
    if zpath.exists():
        try: zpath.unlink()
        except OSError: pass
    comp_const, comp_level = _resolve_zip_options(zip_method, zip_level)
    total_uncompressed = 0
    # Only pass compresslevel if the method supports it
    if comp_level is None:
        zf_ctx = zipfile.ZipFile(zpath, "w", compression=comp_const)
    else:
        zf_ctx = zipfile.ZipFile(zpath, "w", compression=comp_const, compresslevel=comp_level)

    with zf_ctx as zf:
        for rel in sorted(found_paths):
            if rel.replace("\\","/") == zip_name.replace("\\","/"):
                continue
            name = Path(rel).name
            if not include_generated and name in control_names:
                continue
            if not include_code_html and rel in code_targets:
                continue
            ap = root / rel
            try:
                st = ap.stat()
            except OSError:
                continue
            if max_bytes and st.st_size > max_bytes:
                continue
            zf.write(ap, arcname=rel)
            total_uncompressed += int(st.st_size)

    compressed_size = 0
    if zpath.exists():
        with zipfile.ZipFile(zpath, "r") as zf:
            compressed_size = sum(zinfo.compress_size for zinfo in zf.infolist())

    return zpath.exists(), total_uncompressed, compressed_size


# ---------- Main ----------
def main():
    parser = argparse.ArgumentParser(description="Generate files.json and per-folder index pages.")
    parser.add_argument("--root", type=str, default=".", help="Root directory to scan")
    parser.add_argument("--json", type=str, default="files.json", help="Output JSON path (relative to root)")
    parser.add_argument("--html-out", type=str, default="files.html", help="Filename for per-folder index pages")
    parser.add_argument("--exclude", type=str, nargs="*", default=DEFAULT_EXCLUDES, help="Glob patterns to exclude")
    parser.add_argument("--title", type=str, default="Directory Index", help="Title shown in the UI")
    parser.add_argument("--pretty", dest="pretty", action="store_true", default=False, help="Pretty-print JSON")
    parser.add_argument("--no-pretty", dest="pretty", action="store_false", help="Do not pretty-print JSON (default)")

    # Code->HTML options
    parser.add_argument("--code-html", dest="code_html", action="store_true", default=True, help="Enable code-to-HTML rendering (default)")
    parser.add_argument("--no-code-html", dest="code_html", action="store_false", help="Disable code-to-HTML rendering")
    parser.add_argument("--code-ext", type=str, nargs="*", default=DEFAULT_CODE_EXT, help="Extensions treated as code")
    parser.add_argument("--code-max-bytes", type=int, default=DEFAULT_CODE_MAX_BYTES, help="Max source size to render to HTML")
    parser.add_argument("--code-html-out", type=str, default="{relpath}.html", help="Pattern for code HTML outputs (use {relpath},{dir},{name},{stem},{ext})")
    parser.add_argument("--hide-code-html", dest="hide_code_html", action="store_true", default=True,
                    help="Hide generated code-viewer HTML files from the listing (default)")
    parser.add_argument("--no-hide-code-html", dest="hide_code_html", action="store_false",
                    help="Show generated code-viewer HTML files in the listing")

    # Overwrite + embed + listing options
    parser.add_argument("--overwrite", action="store_true", help="Force overwrite of generated HTML files")
    parser.add_argument("--embed-json", dest="embed_json", action="store_true", default=True, help="Embed manifest JSON into each HTML (default)")
    parser.add_argument("--no-embed-json", dest="embed_json", action="store_false", help="Do not embed JSON (fetch files.json at runtime)")
    parser.add_argument("--hide-generated", dest="hide_generated", action="store_true", default=True, help="Hide manifest and per-folder page from the listing (default)")
    parser.add_argument("--no-hide-generated", dest="hide_generated", action="store_false", help="Do not hide generated artifacts")

    # Zip options
    parser.add_argument("--zip", dest="make_zip", action="store_true", default=True,
                        help="Create a single zip archive of all discovered files")
    parser.add_argument("--zip-name", type=str, default="auto",
                        help="Name of the output zip archive (written at root)")
    parser.add_argument("--zip-max-bytes", type=int, default=0,
                        help="Skip files larger than this many bytes (0 = no limit)")
    parser.add_argument("--zip-include-generated", dest="zip_include_generated", action="store_true", default=True,
                        help="Include generated control files (manifest and per-folder page) in the zip (default)")
    parser.add_argument("--no-zip-include-generated", dest="zip_include_generated", action="store_false",
                        help="Exclude manifest/per-folder page from the zip")
    parser.add_argument("--zip-include-code-html", dest="zip_include_code_html", action="store_true", default=True,
                        help="Include generated code-viewer HTML files in the zip (default)")
    parser.add_argument("--no-zip-include-code-html", dest="zip_include_code_html", action="store_false",
                        help="Exclude code-viewer HTML files from the zip")
    parser.add_argument("--zip-method", choices=["deflate","lzma","bzip2"], default="deflate")
    parser.add_argument("--zip-level", type=int, default=9)  # only for deflate/bzip2

    args = parser.parse_args()

    root = Path(args.root).resolve()
    if not root.exists() or not root.is_dir():
        print(f"[!] Root does not exist or is a file: {root}", file=sys.stderr)
        sys.exit(2)

    folder_name = Path(args.root).resolve().name or "root"
    zip_name = args.zip_name if args.zip_name.upper() != "AUTO" else f"{folder_name}.zip"

    code_exts = set((e if e.startswith(".") else "."+e).lower() for e in args.code_ext)

    if args.code_html and not HAVE_PYGMENTS:
        print("[!] Pygments not installed. Plain <pre> will be used. Install: pip install pygments", file=sys.stderr)

    excludes = list(args.exclude) if args.exclude else []
    files, folders, n_code, code_targets, found_paths = scan(
        root=root,
        excludes=excludes,
        code_html=args.code_html,
        code_exts=code_exts,
        max_code_bytes=args.code_max_bytes,
        site_title=args.title,
        code_out_pattern=args.code_html_out,
        overwrite=args.overwrite,
        json_name=Path(args.json).name,
        html_out_name=Path(args.html_out).name,
        hide_generated=args.hide_generated,
        hide_code_html=args.hide_code_html,
    )

    zip_exists = False
    zip_size = 0
    if args.make_zip:
      control_names = { Path(args.json).name, Path(args.html_out).name }
      zip_exists, zip_size_uncomp, zip_size_comp = build_zip(
          root=root,
          found_paths=found_paths,
          zip_name=zip_name,
          include_generated=args.zip_include_generated,
          include_code_html=args.zip_include_code_html,
          code_targets=code_targets,
          control_names=control_names,
          max_bytes=args.zip_max_bytes,
          zip_method=args.zip_method,
          zip_level=args.zip_level,
)
      files = [f for f in files if f["path"] != zip_name] # hide zip on purpose

    total_size = sum(f["size"] for f in files)
    manifest = {
        "version": 0.6,
        "author": "olivier.vitrac@gmail.com",
        "platform": "Generative Simulation Initiative",
        "title": args.title,
        "root": ".",
        "generated_at": datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC"),
        "summary": {
            "files": len(files),
            "folders": len(folders),
            "total_size": total_size,
            "excludes": excludes,
            "code_html_enabled": bool(args.code_html),
            "code_html_rendered": int(n_code),
            "zip_enabled": bool(args.make_zip),
            "zip_name": zip_name if args.make_zip else "",
            "zip_exists": bool(zip_exists),
            "zip_uncompressed": int(zip_size_uncomp),
            "zip_compressed": int(zip_size_comp),
        },
        "files": sorted(files, key=lambda f: f["path"].lower()),
    }


    # Write files.json (even if embedding, keep it for online fetchers)
    json_path = (root / args.json)
    write_json(json_path, manifest, args.pretty)

    # Root + per-folder HTML (named by --html-out)
    created = 0
    root_html = root / args.html_out
    if args.overwrite or not root_html.exists():
        root_html.write_text(build_html(args.title, "/", args.embed_json, manifest if args.embed_json else None), encoding="utf-8")
        created += 1

    for fol in folders:
        if fol == "/":
            continue
        fol_path = root / fol.strip("/")
        out_html = fol_path / args.html_out
        if args.overwrite or not out_html.exists():
            out_html.write_text(build_html(args.title, fol, args.embed_json, manifest if args.embed_json else None), encoding="utf-8")
            created += 1

    # Console summary
    def human(n):
        u = ["B","KB","MB","GB","TB"]; i=0; x=float(n)
        while x>=1024 and i<len(u)-1: x/=1024; i+=1
        return (f"{x:.1f}" if i>0 else f"{int(x)}") + " " + u[i]

    print(f"[OK] Wrote {json_path.relative_to(root)} ({len(files)} files, {len(folders)} folders, {human(total_size)} total)")
    print(f"[OK] Generated HTML pages: {created} (using '{args.html_out}') {'[overwrite]' if args.overwrite else ''}")
    if args.code_html:
        print(f"[OK] Code HTML generated/updated: {n_code} file(s); max size {args.code_max_bytes}; ext set: {', '.join(sorted(code_exts))}")
    print(f"[i ] Excludes: {', '.join(excludes) if excludes else '(none)'}")
    print(f"[i ] Embed JSON in HTML: {args.embed_json}")
    print(f"[i ] Hide generated artifacts in listing: {args.hide_generated}")
    print(f"[i ] Hide code-viewer HTML in listing: {args.hide_code_html}")
    if args.make_zip:
      human_zip = (lambda n: (f"{n/1024/1024:.1f} MB" if n >= 1024*1024 else f"{n/1024:.1f} KB" if n>=1024 else f"{n} B"))(zip_size)
      print(f"[OK] Zip: {args.zip_name} ({'exists' if zip_exists else 'missing'}; {human_zip}; include_generated={args.zip_include_generated}; include_code_html={args.zip_include_code_html})")
    print(f"[→ ] Deploy with surge (e.g., surge ./)")
    if not args.embed_json:
        print(f"[!] Local file:// viewing will need a server or --embed-json")

if __name__ == "__main__":
    # helpful MIME hints
    mimetypes.add_type("application/pdf", ".pdf")
    mimetypes.add_type("text/markdown", ".md")
    mimetypes.add_type("text/jsx", ".jsx")
    mimetypes.add_type("text/tsx", ".tsx")
    mimetypes.add_type("application/wasm", ".wasm")
    main()

Functions

def build_html(title: str, default_folder: str = '/', embed_json: bool = True, manifest: Optional[Dict[str, Any]] = None) ‑> str

Build the HTML; optionally embed the manifest JSON.

Expand source code
def build_html(title: str, default_folder: str = "/", embed_json: bool = True, manifest: Optional[Dict[str,Any]] = None) -> str:
    """Build the HTML; optionally embed the manifest JSON."""
    html = HTML_TEMPLATE.replace("{{TITLE}}", title)
    if embed_json and manifest is not None:
        embed = "<script id=\"manifest\" type=\"application/json\">" + json.dumps(manifest, ensure_ascii=False) + "</script>"
    else:
        embed = ""
    html = html.replace("{{EMBED_MANIFEST}}", embed)
    injection = f"""
<script>
  if (!location.hash) {{
      location.hash = "#folder={default_folder}";
  }}
</script>
"""
    return html.replace("</body>", injection + "\n</body>")
def build_zip(root: Path, found_paths: Set[str], zip_name: str, include_generated: bool, include_code_html: bool, code_targets: Set[str], control_names: Set[str], max_bytes: int, zip_method: str, zip_level: int) ‑> Tuple[bool, int, int]

Create a zip at root/zip_name containing files from found_paths. - Skips the zip itself if already present. - If include_generated is False: skip manifest/per-folder page names. - If include_code_html is False: skip any path in code_targets. - If max_bytes > 0: skip files larger than that threshold. Returns (exists, total_bytes_written).

Expand source code
def build_zip(root: Path,
              found_paths: Set[str],
              zip_name: str,
              include_generated: bool,
              include_code_html: bool,
              code_targets: Set[str],
              control_names: Set[str],
              max_bytes: int,
              zip_method: str,  # "deflate" | "lzma" | "bzip2"
              zip_level: int,   # e.g. 9 (DEFLATE/BZIP2 only)
              ) -> Tuple[bool, int, int]:
    """
    Create a zip at root/zip_name containing files from found_paths.
    - Skips the zip itself if already present.
    - If include_generated is False: skip manifest/per-folder page names.
    - If include_code_html is False: skip any path in code_targets.
    - If max_bytes > 0: skip files larger than that threshold.
    Returns (exists, total_bytes_written).
    """
    comp_const, comp_level = _resolve_zip_options(zip_method, zip_level)
    zpath = root / zip_name
    # Avoid zipping the previous zip
    if zpath.exists():
        try: zpath.unlink()
        except OSError: pass
    comp_const, comp_level = _resolve_zip_options(zip_method, zip_level)
    total_uncompressed = 0
    # Only pass compresslevel if the method supports it
    if comp_level is None:
        zf_ctx = zipfile.ZipFile(zpath, "w", compression=comp_const)
    else:
        zf_ctx = zipfile.ZipFile(zpath, "w", compression=comp_const, compresslevel=comp_level)

    with zf_ctx as zf:
        for rel in sorted(found_paths):
            if rel.replace("\\","/") == zip_name.replace("\\","/"):
                continue
            name = Path(rel).name
            if not include_generated and name in control_names:
                continue
            if not include_code_html and rel in code_targets:
                continue
            ap = root / rel
            try:
                st = ap.stat()
            except OSError:
                continue
            if max_bytes and st.st_size > max_bytes:
                continue
            zf.write(ap, arcname=rel)
            total_uncompressed += int(st.st_size)

    compressed_size = 0
    if zpath.exists():
        with zipfile.ZipFile(zpath, "r") as zf:
            compressed_size = sum(zinfo.compress_size for zinfo in zf.infolist())

    return zpath.exists(), total_uncompressed, compressed_size
def ensure_code_html(src_path: Path, html_path: Path, title: str, max_bytes: int, overwrite: bool) ‑> bool
Expand source code
def ensure_code_html(src_path: Path, html_path: Path, title: str, max_bytes: int, overwrite: bool) -> bool:
    try:
        if src_path.stat().st_size > max_bytes:
            return False
        dst_exists = html_path.exists()
        src_m = src_path.stat().st_mtime
        dst_m = html_path.stat().st_mtime if dst_exists else -1
        need = overwrite or (not dst_exists) or (dst_m <= src_m)   # <= fixes same-second ties
        if need:
            text = src_path.read_text(encoding="utf-8", errors="replace")
            html = highlight_code_to_html(text, src_path.name, title)
            html_path.parent.mkdir(parents=True, exist_ok=True)
            html_path.write_text(html, encoding="utf-8")
            return True
        return False
    except Exception:
        return False
def format_code_out(relpath: str, pattern: str) ‑> str
Expand source code
def format_code_out(relpath: str, pattern: str) -> str:
    p = Path(relpath)
    mapping = {
        "relpath": relpath,
        "dir": p.parent.as_posix(),
        "name": p.name,
        "stem": p.stem,
        "ext": p.suffix,
    }
    return pattern.format(**mapping).lstrip("./")
def highlight_code_to_html(src_text: str, filename: str, title: str) ‑> str

Render code to standalone dark-themed HTML with inline CSS (works on file://).

Expand source code
def highlight_code_to_html(src_text: str, filename: str, title: str) -> str:
    """Render code to standalone dark-themed HTML with inline CSS (works on file://)."""
    def esc(x: str) -> str:
        return x.replace("&","&amp;").replace("<","&lt;").replace(">","&gt;")
    header = (
        "<div style='padding:8px 10px;"
        "background:#0d1218;border-bottom:1px solid #17202a;color:#e7edf4;"
        "font-family:system-ui,Arial,sans-serif;font-size:14px;'>"
        f"<strong>🌱 {esc(title)}</strong> — viewing <code>{esc(filename)}</code></div>"
    )
    base_css = """
      body{margin:0;background:#0b0f14;color:#e7edf4}
      .highlight{background:#0b0f14 !important; color:#e7edf4 !important;}
      pre{margin:0}
      .highlighttable{border-collapse:separate;border-spacing:0;width:100%}
      .highlighttable td{vertical-align:top}
      .linenos{background:#0b0f14;color:#8aa1b4 !important; padding:0 .5em; user-select:none}
      .linenos pre{background:transparent !important; color:inherit !important}
      .code{background:#0b0f14}
    """
    if not HAVE_PYGMENTS:
        body = f"<pre class='highlight' style='padding:12px;overflow:auto'>{esc(src_text)}</pre>"
        return f"<!doctype html><meta charset='utf-8'><title>{esc(title)} · {esc(filename)}</title><style>{base_css}</style>{header}{body}"
    try:
        try:
            lexer = get_lexer_for_filename(filename, stripall=False)
        except Exception:
            lexer = TextLexer(stripall=False)
        formatter = HtmlFormatter(full=False, noclasses=False, linenos=True)
        pyg_css = formatter.get_style_defs('.highlight')
        highlighted = highlight(src_text, lexer, formatter)
        style = f"<style>{base_css}\n{pyg_css}</style>"
        return f"<!doctype html><meta charset='utf-8'><title>{esc(title)} · {esc(filename)}</title>{style}{header}<div style='padding:0 8px 8px'>{highlighted}</div>"
    except Exception:
        body = f"<pre class='highlight' style='padding:12px;overflow:auto'>{esc(src_text)}</pre>"
        return f"<!doctype html><meta charset='utf-8'><title>{esc(title)} · {esc(filename)}</title><style>{base_css}</style>{header}{body}"
def html_escape(s: str) ‑> str
Expand source code
def html_escape(s: str) -> str:
    return s.replace("&","&amp;").replace("<","&lt;").replace(">","&gt;")
def main()
Expand source code
def main():
    parser = argparse.ArgumentParser(description="Generate files.json and per-folder index pages.")
    parser.add_argument("--root", type=str, default=".", help="Root directory to scan")
    parser.add_argument("--json", type=str, default="files.json", help="Output JSON path (relative to root)")
    parser.add_argument("--html-out", type=str, default="files.html", help="Filename for per-folder index pages")
    parser.add_argument("--exclude", type=str, nargs="*", default=DEFAULT_EXCLUDES, help="Glob patterns to exclude")
    parser.add_argument("--title", type=str, default="Directory Index", help="Title shown in the UI")
    parser.add_argument("--pretty", dest="pretty", action="store_true", default=False, help="Pretty-print JSON")
    parser.add_argument("--no-pretty", dest="pretty", action="store_false", help="Do not pretty-print JSON (default)")

    # Code->HTML options
    parser.add_argument("--code-html", dest="code_html", action="store_true", default=True, help="Enable code-to-HTML rendering (default)")
    parser.add_argument("--no-code-html", dest="code_html", action="store_false", help="Disable code-to-HTML rendering")
    parser.add_argument("--code-ext", type=str, nargs="*", default=DEFAULT_CODE_EXT, help="Extensions treated as code")
    parser.add_argument("--code-max-bytes", type=int, default=DEFAULT_CODE_MAX_BYTES, help="Max source size to render to HTML")
    parser.add_argument("--code-html-out", type=str, default="{relpath}.html", help="Pattern for code HTML outputs (use {relpath},{dir},{name},{stem},{ext})")
    parser.add_argument("--hide-code-html", dest="hide_code_html", action="store_true", default=True,
                    help="Hide generated code-viewer HTML files from the listing (default)")
    parser.add_argument("--no-hide-code-html", dest="hide_code_html", action="store_false",
                    help="Show generated code-viewer HTML files in the listing")

    # Overwrite + embed + listing options
    parser.add_argument("--overwrite", action="store_true", help="Force overwrite of generated HTML files")
    parser.add_argument("--embed-json", dest="embed_json", action="store_true", default=True, help="Embed manifest JSON into each HTML (default)")
    parser.add_argument("--no-embed-json", dest="embed_json", action="store_false", help="Do not embed JSON (fetch files.json at runtime)")
    parser.add_argument("--hide-generated", dest="hide_generated", action="store_true", default=True, help="Hide manifest and per-folder page from the listing (default)")
    parser.add_argument("--no-hide-generated", dest="hide_generated", action="store_false", help="Do not hide generated artifacts")

    # Zip options
    parser.add_argument("--zip", dest="make_zip", action="store_true", default=True,
                        help="Create a single zip archive of all discovered files")
    parser.add_argument("--zip-name", type=str, default="auto",
                        help="Name of the output zip archive (written at root)")
    parser.add_argument("--zip-max-bytes", type=int, default=0,
                        help="Skip files larger than this many bytes (0 = no limit)")
    parser.add_argument("--zip-include-generated", dest="zip_include_generated", action="store_true", default=True,
                        help="Include generated control files (manifest and per-folder page) in the zip (default)")
    parser.add_argument("--no-zip-include-generated", dest="zip_include_generated", action="store_false",
                        help="Exclude manifest/per-folder page from the zip")
    parser.add_argument("--zip-include-code-html", dest="zip_include_code_html", action="store_true", default=True,
                        help="Include generated code-viewer HTML files in the zip (default)")
    parser.add_argument("--no-zip-include-code-html", dest="zip_include_code_html", action="store_false",
                        help="Exclude code-viewer HTML files from the zip")
    parser.add_argument("--zip-method", choices=["deflate","lzma","bzip2"], default="deflate")
    parser.add_argument("--zip-level", type=int, default=9)  # only for deflate/bzip2

    args = parser.parse_args()

    root = Path(args.root).resolve()
    if not root.exists() or not root.is_dir():
        print(f"[!] Root does not exist or is a file: {root}", file=sys.stderr)
        sys.exit(2)

    folder_name = Path(args.root).resolve().name or "root"
    zip_name = args.zip_name if args.zip_name.upper() != "AUTO" else f"{folder_name}.zip"

    code_exts = set((e if e.startswith(".") else "."+e).lower() for e in args.code_ext)

    if args.code_html and not HAVE_PYGMENTS:
        print("[!] Pygments not installed. Plain <pre> will be used. Install: pip install pygments", file=sys.stderr)

    excludes = list(args.exclude) if args.exclude else []
    files, folders, n_code, code_targets, found_paths = scan(
        root=root,
        excludes=excludes,
        code_html=args.code_html,
        code_exts=code_exts,
        max_code_bytes=args.code_max_bytes,
        site_title=args.title,
        code_out_pattern=args.code_html_out,
        overwrite=args.overwrite,
        json_name=Path(args.json).name,
        html_out_name=Path(args.html_out).name,
        hide_generated=args.hide_generated,
        hide_code_html=args.hide_code_html,
    )

    zip_exists = False
    zip_size = 0
    if args.make_zip:
      control_names = { Path(args.json).name, Path(args.html_out).name }
      zip_exists, zip_size_uncomp, zip_size_comp = build_zip(
          root=root,
          found_paths=found_paths,
          zip_name=zip_name,
          include_generated=args.zip_include_generated,
          include_code_html=args.zip_include_code_html,
          code_targets=code_targets,
          control_names=control_names,
          max_bytes=args.zip_max_bytes,
          zip_method=args.zip_method,
          zip_level=args.zip_level,
)
      files = [f for f in files if f["path"] != zip_name] # hide zip on purpose

    total_size = sum(f["size"] for f in files)
    manifest = {
        "version": 0.6,
        "author": "olivier.vitrac@gmail.com",
        "platform": "Generative Simulation Initiative",
        "title": args.title,
        "root": ".",
        "generated_at": datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC"),
        "summary": {
            "files": len(files),
            "folders": len(folders),
            "total_size": total_size,
            "excludes": excludes,
            "code_html_enabled": bool(args.code_html),
            "code_html_rendered": int(n_code),
            "zip_enabled": bool(args.make_zip),
            "zip_name": zip_name if args.make_zip else "",
            "zip_exists": bool(zip_exists),
            "zip_uncompressed": int(zip_size_uncomp),
            "zip_compressed": int(zip_size_comp),
        },
        "files": sorted(files, key=lambda f: f["path"].lower()),
    }


    # Write files.json (even if embedding, keep it for online fetchers)
    json_path = (root / args.json)
    write_json(json_path, manifest, args.pretty)

    # Root + per-folder HTML (named by --html-out)
    created = 0
    root_html = root / args.html_out
    if args.overwrite or not root_html.exists():
        root_html.write_text(build_html(args.title, "/", args.embed_json, manifest if args.embed_json else None), encoding="utf-8")
        created += 1

    for fol in folders:
        if fol == "/":
            continue
        fol_path = root / fol.strip("/")
        out_html = fol_path / args.html_out
        if args.overwrite or not out_html.exists():
            out_html.write_text(build_html(args.title, fol, args.embed_json, manifest if args.embed_json else None), encoding="utf-8")
            created += 1

    # Console summary
    def human(n):
        u = ["B","KB","MB","GB","TB"]; i=0; x=float(n)
        while x>=1024 and i<len(u)-1: x/=1024; i+=1
        return (f"{x:.1f}" if i>0 else f"{int(x)}") + " " + u[i]

    print(f"[OK] Wrote {json_path.relative_to(root)} ({len(files)} files, {len(folders)} folders, {human(total_size)} total)")
    print(f"[OK] Generated HTML pages: {created} (using '{args.html_out}') {'[overwrite]' if args.overwrite else ''}")
    if args.code_html:
        print(f"[OK] Code HTML generated/updated: {n_code} file(s); max size {args.code_max_bytes}; ext set: {', '.join(sorted(code_exts))}")
    print(f"[i ] Excludes: {', '.join(excludes) if excludes else '(none)'}")
    print(f"[i ] Embed JSON in HTML: {args.embed_json}")
    print(f"[i ] Hide generated artifacts in listing: {args.hide_generated}")
    print(f"[i ] Hide code-viewer HTML in listing: {args.hide_code_html}")
    if args.make_zip:
      human_zip = (lambda n: (f"{n/1024/1024:.1f} MB" if n >= 1024*1024 else f"{n/1024:.1f} KB" if n>=1024 else f"{n} B"))(zip_size)
      print(f"[OK] Zip: {args.zip_name} ({'exists' if zip_exists else 'missing'}; {human_zip}; include_generated={args.zip_include_generated}; include_code_html={args.zip_include_code_html})")
    print(f"[→ ] Deploy with surge (e.g., surge ./)")
    if not args.embed_json:
        print(f"[!] Local file:// viewing will need a server or --embed-json")
def match_any(patterns: Iterable[str], name: str) ‑> bool
Expand source code
def match_any(patterns: Iterable[str], name: str) -> bool:
    for p in patterns:
        if fnmatch.fnmatch(name, p):
            return True
    return False
def scan(root: Path, excludes: List[str], code_html: bool, code_exts: Set[str], max_code_bytes: int, site_title: str, code_out_pattern: str, overwrite: bool, json_name: str, html_out_name: str, hide_generated: bool, hide_code_html: bool) ‑> Tuple[List[Dict[str, Any]], Set[str], int, Set[str], Set[str]]

Scan files, optionally emit code HTML, return (files, folders, n_code_html_written). Returns: files (for manifest table), folders, n_code_html_written, code_targets (all viewer paths), found_paths (all discovered files)

Expand source code
def scan(root: Path,
         excludes: List[str],
         code_html: bool,
         code_exts: Set[str],
         max_code_bytes: int,
         site_title: str,
         code_out_pattern: str,
         overwrite: bool,
         json_name: str,
         html_out_name: str,
         hide_generated: bool,
         hide_code_html: bool
         ) -> Tuple[List[Dict[str,Any]], Set[str], int, Set[str], Set[str]]:
    """
    Scan files, optionally emit code HTML, return (files, folders, n_code_html_written).
    Returns: files (for manifest table), folders, n_code_html_written,
          code_targets (all viewer paths), found_paths (all discovered files)
    """
    files: List[Dict[str,Any]] = []
    folders: Set[str] = set(["/"])
    made = 0
    code_targets: Set[str] = set()
    found_paths: Set[str] = set()

    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if not should_skip(Path(dirpath)/d, excludes, root)]

        rel_dir = Path(dirpath).relative_to(root).as_posix()
        if rel_dir != ".":
            acc = ""
            for seg in rel_dir.split("/"):
                acc += "/" + seg
                folders.add(acc)

        for fn in filenames:
            p = Path(dirpath) / fn
            if should_skip(p, excludes, root):
                continue

            try:
                st = p.stat()
            except OSError:
                continue

            size = int(st.st_size)
            mtime_epoch = int(st.st_mtime)
            rel_path = p.relative_to(root).as_posix()
            name = Path(rel_path).name
            ext = Path(fn).suffix
            mime = mimetypes.guess_type(fn, strict=False)[0] or ""

            # record each file (for zip)
            found_paths.add(rel_path)

            # Never convert HTML-like or our own generated control files
            is_htmlish = rel_path.lower().endswith(".html")
            is_manifest = (name == json_name)
            is_index_like = (name == html_out_name)

            # Optionally hide generated artifacts from the listing
            if hide_generated and (is_manifest or is_index_like):
                pass_hidden = True
            else:
                pass_hidden = False

            href: Optional[str] = None

            # Conversion decision
            if code_html:
                if (ext.lower() in code_exts) and (not is_htmlish) and (not is_manifest) and (not is_index_like):
                    out_rel = format_code_out(rel_path, code_out_pattern)
                    # Avoid self-mapping and infinite chains
                    if out_rel != rel_path and not out_rel.lower().endswith(".html.html"):
                        code_targets.add(out_rel)
                        out_abs = (root / out_rel)
                        if ensure_code_html(p, out_abs, site_title, max_code_bytes, overwrite):
                            made += 1
                        href = out_rel

            file_entry = {
                "path": rel_path,               # original path
                "href": href or rel_path,       # target to open
                "name": fn,
                "ext": ext if ext else "",
                "size": size,
                "mtime_iso": datetime.fromtimestamp(mtime_epoch, tz=timezone.utc).isoformat(),
                "mtime_epoch": mtime_epoch,
                "mime": mime,
            }

            if not pass_hidden:
                files.append(file_entry)

    if hide_code_html and code_targets:
      files = [f for f in files if f["path"] not in code_targets]
    return files, folders, made, code_targets, found_paths
def should_skip(path: Path, excludes: List[str], root: Path) ‑> bool
Expand source code
def should_skip(path: Path, excludes: List[str], root: Path) -> bool:
    rel = path.relative_to(root).as_posix()
    parts = rel.split("/")
    if match_any(excludes, rel): return True
    if match_any(excludes, path.name): return True
    for seg in parts:
        if match_any(excludes, seg): return True
    return False
def write_json(path: Path, payload: Dict[str, Any], pretty: bool)
Expand source code
def write_json(path: Path, payload: Dict[str,Any], pretty: bool):
    if pretty:
        path.write_text(json.dumps(payload, indent=2, ensure_ascii=False), encoding="utf-8")
    else:
        path.write_text(json.dumps(payload, separators=(",",":"), ensure_ascii=False), encoding="utf-8")