The Macroscope Collaboratory

@techreport{hamilton2026macroscope, author = {Hamilton, Michael P., Ph.D.}, title = {The Macroscope Collaboratory}, institution = {Canemah Nature Laboratory}, year = {2026}, number = {CNL-TN-2026-047}, month = {april}, url = {https://canemah.org/archive/document.php?id=CNL-TN-2026-047}, abstract = {Versions 0.1 and 0.2 of this technical note documented the first-light test of the Investigation Wizard as a PHP in-request application driving the Anthropic API directly, with a seven-phase workflow, ecological context priors, and the STR-001 diurnal temperature range investigation. That test validated the scientific workflow (phases, notebook-as-memory, investigator oversight as error correction) but exposed operational limits of the implementation substrate: per-token API costs, fragile in-request LLM orchestration, and an impedance mismatch between the synchronous Apache request lifecycle and the long-running, multi-step, tool-calling nature of real investigations. This revision documents the architectural pivot made on April 10, 2026 in response to that analysis. The Collaboratory is being rebuilt around three load-bearing components that did not exist (or were not yet integrated) at the time of the STR-001 test: the Macroscope worker framework (a filesystem job queue under launchd), a Model Context Protocol (MCP) server exposing the worker framework to Claude Desktop as tool calls, and Claude Desktop itself as the LLM host. The economic consequence of this shift is that investigation reasoning moves from per-token API billing onto the author's existing Max subscription, making extended exploratory work affordable. The architectural consequence is cleaner separation of concerns: the LLM lives in Claude Desktop, the execution substrate lives in the worker framework, the persistence layer lives in a new `collaboratory\_db` database, and the user-facing UI reduces to a thin PHP front-end. First-light testing of the new substrate was conducted in this session and is reported below. The Collaboratory MCP server (`Projects/Workbench/Collaboratory/mcp/`) was built from scratch in six files under 200 lines each, verified in phases 1 through 5 (layout, Python environment, static imports, direct Python enqueue with PHP dispatcher pickup, FastMCP launch with curl), pivoted from SSE to stdio transport after discovering Claude Desktop's HTTPS-only constraint on its custom-connector UI, registered in `claude\_desktop\_config.json` as a sibling of the existing `preferences` object, and exercised end-to-end with the echo worker from within this Cowork conversation. The round trip (enqueue → processing → done → result retrieval) completed successfully, confirming that the MCP transport, the Python enqueue library, the atomic file rename, the PHP dispatcher tick, the worker contract, and the result-readback path are all operational and reachable from Claude. With the substrate in place, forward work is sequenced into phases A through F: build the `web\_search` worker following the MNG Ollama enrichment pattern, instantiate `collaboratory\_db`, add investigation-state MCP tools, build the minimal PHP wizard UI, run the first real investigation, and optionally add an in-wizard AI assistant pane. This note ends with the SQL for the initial `collaboratory\_db` schema and the worker scaffold as delivered.} }