hacka.ai

TouchFS - the Swiss Army Knife of Synthetic Data Generation

I started prompting/vibe coding TouchFS late December of 2024 to pressure test Anthropic's latest frontier model Claude-3.5-Sonnet, which suddenly scored really well on a particular benchmark SWE-Bench Verified, containing real-world software engineering problems.

Anthropic's announcement of the Model Context Protocol (MCP) also happened a month prior.

Fun fact: the one major hallucination event experienced during long hours of agentic coding, where the model completely misunderstands something, completes the hallucinated implementation and writes tests to validate the hallucinations? I asked for a generally MCP-compatible architecture, meaning MCP as in Model Context Protocol. This is one type of hallucination to be aware of.

I referred to a paper not yet released, although training data up until the time of training data cut-off may very well contain both discussions, and quite possibly more than one implementation of earlier versions/proposals for MCP. Anthropic uses proprietary training data which is at least quite likely to contain bits and pieces of what became their MCP paper released by the 25:th of November 2024. If sufficiently many ducks are in a row, the model will just hallucinate the rest.

I did not notice, just kept on building.

The Concept

At its core, TouchFS was an experimental prototype to generate context-aware content directly through the lowest-layered operations performed by processes on a file system.

This would allow for quite interesting usage patterns where essentially arbitrary file system events (file created, read, updated, deleted, flag changed etc.) could be used as logic for suddenly generating context-aware content in-process, without any third party interactions from other processes or programs.

One could even imagine SSHFS-mounting a remote file system with these properties, and simply opening test.sh in a folder containing some script, will then have test.sh almost magically contain tests matching the other program. On first touch, so to speak.

Which also lead to the gimmicky idea to have content generation being triggered specifically by the ancient unix-command `touch`, which otherwise simply create empty files and/or updates their time stamps.

Why not re-purpose it so that touching a file also fills it with appropriate content, using context from adjacent files? Works for text, code and even images in the current implementation.

Anyway, that was the experiment. An amazing amount of code was produced, with great test coverage, to turn this idea into an actual implementation in such a short time; the git commit history tells the entire story.

One interesting aspect of TouchFS is its ability to block file operations while generating content from context in adjacent files — a feature that makes content seemingly appear spontaneously out of thin air, in-process, as the actual read syscall waits until content is generated by the file system driver (TouchFS).

These properties distinguishes TouchFS from MCP, where MCP - as the name entails - is a protocol.

# A simple demonstration of the concept
touchfs mount workspace -F "Create a web application"

# Creating files that are aware of each other
touch workspace/README.md
touch workspace/app.py

Experimental Mechanics

The order in which files are created affected their generated content, with each unique context (including generation order) producing different results.

# Scenario: README first, then app
touch workspace/README.md
touch workspace/app.py

# Result: The app implementation is influenced by the README content
# ┌─────────────────┐          ┌─────────────────┐
# │   README.md     │          │     app.py      │
# │ (Generated 1st) │──────────│  (Generated 2nd) │
# │                 │          │                  │
# │ Web Scraper     │          │ import requests  │
# │ ============    │  shapes  │                  │
# │ A Python tool   │───app────│ def scrape():   │
# │ for scraping    │  design  │   # Implement   │
# │ websites...     │          │   # scraping    │
# └─────────────────┘          └─────────────────┘

Use Cases Explored

Technical Implementation

The experimental implementation uses command interception, though the concept itself isn't tied to any specific interface. Touchfs leverages libfuse2 to create an overlay file system whereby file system operations can be blocked while generating content. The project is released under the MIT license to allow others to build upon or adapt the ideas it explored.

AI-Generated Podcast

Below is an AI-generated podcast about TouchFS created by NotebookLM. While informative, please note that it's a bit over-hyped compared to the experimental nature of the project described above.

Resources

Those interested in exploring content generation at file system level can find more information at:

GitHub Repository | touchfs.io