redis-fs

Category: Databases & Storage | Uploader: rowantrollope | Downloads: 0 | Version: v1.0（Latest）

Redis-FS is a Redis module that provides a complete POSIX-like filesystem as a native data type, plus a FUSE mount that exposes it as a regular local directory. One Redis key holds one filesystem volume — directories, files, symlinks, permissions, and all metadata.

Changelog: Source: GitHub https://github.com/rowantrollope/redis-fs

Directory Structure

Current level: tree/main/

📁 cli/
- 📁 cmd/
  - 📁 redis-qmd/
    
    📄 main.go 3.1 KB
- 📁 qmd/
  - 📄 dsl.go 4.6 KB
  - 📄 dsl_test.go 1009 B
  - 📄 search.go 7.5 KB
  - 📄 types.go 747 B
- 📄 cloud.go 27.5 KB
- 📄 cloud_test.go 10.0 KB
- 📄 go.mod 441 B
- 📄 go.sum 2.7 KB
- 📄 main.go 41.8 KB
- 📄 Makefile 169 B
- 📄 migration_ignore.go 1.1 KB
- 📄 migration_ignore_test.go 5.0 KB
- 📄 migration_pipeline.go 13.4 KB
- 📄 migration_pipeline_test.go 1.4 KB
- 📄 mount_backend.go 9.8 KB
- 📄 redis_options.go 560 B
- 📄 redis_options_test.go 959 B
- 📄 stat_darwin.go 268 B
- 📄 stat_linux.go 233 B
- 📄 ui.go 7.5 KB
📁 examples/
- 📄 claude_desktop_config.json 155 B
- 📄 codex-settings-migration.md 3.9 KB
📁 mcp_server/
- 📄 __init__.py 147 B
- 📄 server.py 15.1 KB
📁 module/
- 📄 fs.c 115.8 KB
- 📄 fs.h 4.1 KB
- 📄 Makefile 863 B
- 📄 path.c 8.2 KB
- 📄 path.h 1.3 KB
- 📄 redismodule.h 109.6 KB
📁 mount/
- 📁 client/
  - 📄 client.go 441 B
- 📁 cmd/
  - 📁 redis-fs-mount/
    
    📄 main.go 3.9 KB
  - 📁 redis-fs-nfs/
    
    📄 main.go 3.7 KB
- 📁 internal/
  - 📁 cache/
    
    📄 cache.go 1.5 KB
  - 📁 client/
    
    📄 client.go 2.6 KB
    
    📄 glob.go 2.4 KB
    
    📄 glob_test.go 1.9 KB
    
    📄 keys.go 884 B
    
    📄 native.go 40.3 KB
    
    📄 native_test.go 16.6 KB
    
    📄 parser.go 3.3 KB
  - 📁 nfsfs/
    
    📄 fs.go 9.6 KB
  - 📁 redisconn/
    
    📄 redisconn.go 671 B
    
    📄 redisconn_test.go 933 B
  - 📁 redisfs/
    
    📄 attr.go 1001 B
    
    📄 dir.go 4.6 KB
    
    📄 errors.go 1.3 KB
    
    📄 errors_test.go 865 B
    
    📄 file.go 4.4 KB
    
    📄 fs.go 5.8 KB
    
    📄 fs_cache_test.go 1.0 KB
    
    📄 handle.go 2.5 KB
    
    📄 symlink.go 1.2 KB
- 📄 go.mod 637 B
- 📄 go.sum 3.4 KB
- 📄 Makefile 249 B
📁 redis_fs/
- 📄 __init__.py 404 B
- 📄 cli.py 8.1 KB
- 📄 client.py 10.5 KB
- 📄 exceptions.py 574 B
- 📄 py.typed 1 B
📁 redisclaw/
- 📁 redisclaw/
  - 📄 __init__.py 87 B
  - 📄 agent.py 14.0 KB
  - 📄 cli.py 16.5 KB
  - 📄 memory.py 8.3 KB
  - 📄 tools.py 13.5 KB
- 📁 tests/
  - 📄 __init__.py 24 B
  - 📄 test_e2e.py 16.1 KB
- 📄 Makefile 331 B
- 📄 pyproject.toml 605 B
- 📄 README.md 5.6 KB
📁 sandbox/
- 📁 cmd/
  - 📁 sandbox/
    
    📄 main.go 1.7 KB
  - 📁 sandbox-cli/
    
    📄 main.go 3.5 KB
- 📁 internal/
  - 📁 api/
    
    📄 mcp.go 4.2 KB
    
    📄 mcp_tools.go 2.3 KB
    
    📄 server.go 3.9 KB
  - 📁 executor/
    
    📄 monitor.go 3.7 KB
    
    📄 process.go 3.4 KB
- 📄 docker-compose.yml 819 B
- 📄 Dockerfile 1.5 KB
- 📄 entrypoint.sh 1.8 KB
- 📄 go.mod 119 B
- 📄 go.sum 326 B
- 📄 Makefile 268 B
📁 skills/
- 📁 codex-settings-sync/
  - 📁 assets/
    
    📄 .rfsignore 127 B
  - 📄 SKILL.md 2.8 KB
- 📁 redis-fs/
  - 📄 SKILL.md 5.5 KB
- 📄 .DS_Store 6.0 KB
📁 tasks/
- 📄 001_AGENT_SKILL.md 3.4 KB
- 📄 002_PYTHON_LIBRARY.md 6.4 KB
- 📄 003_MCP_SERVER.md 4.7 KB
📁 tests/
- 📁 bench/
  - 📄 go.mod 22 B
  - 📄 main.go 21.6 KB
- 📄 __init__.py 23 B
- 📄 append.py 1.8 KB
- 📄 bench_qmd_suite.py 16.6 KB
- 📄 bench_search.py 3.3 KB
- 📄 chmod.py 1.1 KB
- 📄 chown.py 991 B
- 📄 cp.py 1.7 KB
- 📄 create-test-memories 4.4 KB
- 📄 deletelines.py 4.0 KB
- 📄 echo_cat.py 2.9 KB
- 📄 error_handling.py 4.9 KB
- 📄 find.py 1.5 KB
- 📄 glob_patterns.py 1.6 KB
- 📄 grep.py 1.9 KB
- 📄 hardening_phase0.py 3.4 KB
- 📄 head.py 2.7 KB
- 📄 info.py 888 B
- 📄 insert.py 3.5 KB
- 📄 invariants.py 1.3 KB
- 📄 lifecycle.py 1.3 KB
- 📄 lines.py 3.3 KB
- 📄 ls.py 1.6 KB
- 📄 mkdir.py 1.3 KB
- 📄 mv.py 2.0 KB
- 📄 path_normalize.py 2.2 KB
- 📄 rdb_persistence.py 2.9 KB
- 📄 replace.py 4.5 KB
- 📄 rm.py 1.9 KB
- 📄 stat.py 1.1 KB
- 📄 symlinks.py 3.8 KB
- 📄 tail.py 3.2 KB
- 📄 test_cmd.py 1.3 KB
- 📄 test_mcp_server.py 827 B
- 📄 test_redis_fs.py 8.1 KB
- 📄 touch.py 1.4 KB
- 📄 tree.py 1.2 KB
- 📄 truncate_utimens.py 1.9 KB
- 📄 wc.py 3.2 KB
- 📄 wrong_key_type.py 3.7 KB
📁 third_party/
- 📁 go-nfs/
  - 📁 .github/
    
    📁 workflows/
    
    📄 codeql-analysis.yml 1.5 KB
    
    📄 go.yml 612 B
    
    📄 dependabot.yml 463 B
  - 📁 example/
    
    📁 helloworld/
    
    📄 main.go 1.2 KB
    
    📁 osnfs/
    
    📄 changeos.go 911 B
    
    📄 changeos_unix.go 748 B
    
    📄 main.go 760 B
    
    📁 osview/
    
    📄 main.go 730 B
  - 📁 file/
    
    📄 file.go 405 B
    
    📄 file_other.go 213 B
    
    📄 file_unix.go 469 B
    
    📄 file_wasm.go 288 B
    
    📄 file_windows.go 231 B
  - 📁 helpers/
    
    📁 memfs/
    
    📄 memfs.go 8.0 KB
    
    📄 storage.go 4.0 KB
    
    📄 cachinghandler.go 5.1 KB
    
    📄 nullauthhandler.go 1.6 KB
  - 📄 conn.go 6.9 KB
  - 📄 CONTRIBUTING.md 509 B
  - 📄 errors.go 6.0 KB
  - 📄 file.go 8.5 KB
  - 📄 filesystem.go 316 B
  - 📄 go.mod 770 B
  - 📄 go.sum 4.0 KB
  - 📄 handler.go 1.7 KB
  - 📄 LICENSE 11.1 KB
  - 📄 log.go 4.4 KB
  - 📄 mount.go 1.4 KB
  - 📄 mountinterface.go 1.9 KB
  - 📄 nfs.go 2.2 KB
  - 📄 nfs_onaccess.go 1.1 KB
  - 📄 nfs_oncommit.go 1.4 KB
  - 📄 nfs_oncreate.go 3.4 KB
  - 📄 nfs_onfsinfo.go 2.4 KB
  - 📄 nfs_onfsstat.go 1.4 KB
  - 📄 nfs_ongetattr.go 1.0 KB
  - 📄 nfs_onlink.go 2.5 KB
  - 📄 nfs_onlookup.go 2.3 KB
  - 📄 nfs_onmkdir.go 2.4 KB
  - 📄 nfs_onmknod.go 4.1 KB
  - 📄 nfs_onpathconf.go 1.3 KB
  - 📄 nfs_onread.go 2.2 KB
  - 📄 nfs_onreaddir.go 4.7 KB
  - 📄 nfs_onreaddirplus.go 4.0 KB
  - 📄 nfs_onreadlink.go 1.3 KB
  - 📄 nfs_onremove.go 2.0 KB
  - 📄 nfs_onrename.go 3.0 KB
  - 📄 nfs_onrmdir.go 150 B
  - 📄 nfs_onsetattr.go 1.9 KB
  - 📄 nfs_onsymlink.go 2.4 KB
  - 📄 nfs_onwrite.go 2.9 KB
  - 📄 nfs_test.go 7.7 KB
  - 📄 nfsinterface.go 4.6 KB
  - 📄 README.md 3.2 KB
  - 📄 SECURITY.md 444 B
  - 📄 server.go 2.3 KB
  - 📄 time.go 806 B
📄 .DS_Store 6.0 KB
📄 .gitignore 498 B
📄 AGENTS.md 3.3 KB
📄 BACKLOG.md 3.6 KB
📄 CLAUDE.md 3.3 KB
📄 docker-compose.yml 738 B
📄 Dockerfile 1.3 KB
📄 Dockerfile.mcp 392 B
📄 Dockerfile.redis 401 B
📄 dump.rdb 660.5 KB
📄 LICENSE 33.7 KB
📄 Makefile 1.7 KB
📄 pyproject.toml 1.2 KB
📄 README.md 35.3 KB
📄 redis.log 2.1 KB
📄 redis2.log 1.1 KB
📄 redis_test.log 2.3 KB
📄 RELEASE_NOTES.md 1.2 KB
📄 rfs.config.json 631 B
📄 SKILL.md 9.8 KB
📄 test.json 458 B
📄 test.py 7.2 KB

SKILL.md

# Redis-FS

Redis-FS is a Redis module that provides a complete POSIX-like filesystem as a native data type, plus a FUSE mount that exposes it as a regular local directory. One Redis key holds one filesystem volume — directories, files, symlinks, permissions, and all metadata.

**Why Redis-FS:**
- **Persistence** — data survives process restarts via RDB/AOF
- **Multi-client access** — any Redis client can read/write the same volume concurrently
- **Atomic operations** — every FS.* command is atomic, no partial writes
- **No local disk dependency** — agents can run statelessly; storage lives in Redis
- **Instant cleanup** — `DEL vol` removes an entire filesystem in one command

**Key concept:** one Redis key = one filesystem volume. The key name is the volume name. All paths within a volume are absolute (start with `/`).

## Build

```bash
cd /home/ubuntu/git/redis-fs
make          # builds module/fs.so + mount/redis-fs-mount + rfs
make module   # builds only the Redis module (module/fs.so)
make mount    # builds only the FUSE mount binary
make cli      # builds only the rfs CLI
```

## CLI Commands

The `rfs` binary is at the repo root after building.

| Command | Description |
|---------|-------------|
| `rfs setup` | Interactive first-time wizard — saves config, starts services |
| `rfs up` | Start services from saved config |
| `rfs down` | Stop all services and unmount |
| `rfs status` | Show current status |
| `rfs migrate <dir>` | Import a directory into Redis, honoring `.rfsignore` if present |

Use `--config <path>` before any command to override the config file location:
```bash
./rfs --config /path/to/custom.json up
```

## Programmatic Setup (for agents)

Skip the interactive wizard by writing the config file directly.

### 1. Build

```bash
cd /home/ubuntu/git/redis-fs && make
```

### 2. Write config

```bash
cat > /home/ubuntu/git/redis-fs/rfs.config.json << 'EOF'
{
  "useExistingRedis": false,
  "redisAddr": "localhost:6379",
  "redisPassword": "",
  "redisDB": 0,
  "redisKey": "myfs",
  "mountpoint": "/home/ubuntu/redis-fs",
  "readOnly": false,
  "allowOther": false,
  "redisServerBin": "",
  "modulePath": "",
  "mountBin": "",
  "redisLog": "/tmp/rfs-redis.log",
  "mountLog": "/tmp/rfs-mount.log"
}
EOF
```

Empty strings for `redisServerBin`, `modulePath`, and `mountBin` are auto-detected at runtime.

### 3. Start

```bash
./rfs up
```

### 4. Verify

```bash
./rfs status
ls ~/redis-fs
```

## Configuration Reference

File: `rfs.config.json` (next to the `rfs` binary in the repo root)

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `useExistingRedis` | bool | `false` | `true` = connect to running Redis; `false` = start a managed instance |
| `redisAddr` | string | `"localhost:6379"` | Redis server address (host:port) |
| `redisPassword` | string | `""` | Redis password (empty = no auth) |
| `redisDB` | int | `0` | Redis database number (0–15) |
| `redisKey` | string | `"myfs"` | Redis key name for the filesystem |
| `mountpoint` | string | | Local directory where the filesystem is mounted |
| `readOnly` | bool | `false` | Mount as read-only |
| `allowOther` | bool | `false` | Allow other system users to access the mount |
| `redisServerBin` | string | auto | Path to `redis-server` (only used when `useExistingRedis` is `false`) |
| `modulePath` | string | auto | Path to `fs.so` module |
| `mountBin` | string | auto | Path to `redis-fs-mount` binary |
| `redisLog` | string | `"/tmp/rfs-redis.log"` | Log file for managed Redis |
| `mountLog` | string | `"/tmp/rfs-mount.log"` | Log file for mount daemon |

### Common config patterns

**Managed Redis (simplest):**
```json
{
  "useExistingRedis": false,
  "redisKey": "myfs",
  "mountpoint": "/home/ubuntu/data"
}
```

**Connect to existing Redis:**
```json
{
  "useExistingRedis": true,
  "redisAddr": "redis.local:6379",
  "redisPassword": "secret",
  "redisKey": "myfs",
  "mountpoint": "/home/ubuntu/data"
}
```

## Runtime State

- Config: `rfs.config.json` (repo root, next to the binary)
- State: `~/.rfs/state.json` (runtime PIDs, created/removed automatically)

## FS.* Command Reference

All commands use the pattern: `FS.<CMD> <key> <path> [args...]`

Replace `vol` below with your chosen key name.

### Quick reference

| Unix command | Redis-FS equivalent | Notes |
|---|---|---|
| `echo "text" > file` | `FS.ECHO vol /file "text"` | Creates parents automatically |
| `echo "text" >> file` | `FS.ECHO vol /file "text" APPEND` | Also `FS.APPEND` |
| `cat file` | `FS.CAT vol /file` | Follows symlinks |
| `touch file` | `FS.TOUCH vol /file` | Creates empty file or updates mtime |
| `mkdir dir` | `FS.MKDIR vol /dir` | Parent must exist |
| `mkdir -p a/b/c` | `FS.MKDIR vol /a/b/c PARENTS` | Creates intermediates |
| `ls dir` | `FS.LS vol /dir` | Returns child names |
| `ls -l dir` | `FS.LS vol /dir LONG` | Includes type, mode, size, mtime |
| `rm file` | `FS.RM vol /file` | Works on files, dirs, symlinks |
| `rm -r dir` | `FS.RM vol /dir RECURSIVE` | Deletes entire subtree |
| `cp src dst` | `FS.CP vol /src /dst` | Files only without RECURSIVE |
| `cp -r src dst` | `FS.CP vol /src /dst RECURSIVE` | Deep copy with metadata |
| `mv src dst` | `FS.MV vol /src /dst` | Moves entire subtrees atomically |
| `find dir -name "*.txt"` | `FS.FIND vol /dir "*.txt"` | Glob: `*`, `?`, `[a-z]`, `[!x]`, `\` |
| `find dir -name "*.txt" -type f` | `FS.FIND vol /dir "*.txt" TYPE file` | Filter: file, dir, symlink |
| `grep -r "pattern" dir` | `FS.GREP vol /dir "*pattern*"` | Glob on each line |
| `grep -ri "pattern" dir` | `FS.GREP vol /dir "*pattern*" NOCASE` | Case-insensitive |
| `stat file` | `FS.STAT vol /file` | type, mode, uid, gid, size, times |
| `test -e file` | `FS.TEST vol /file` | Returns 1 or 0 |
| `chmod 0755 file` | `FS.CHMOD vol /file 0755` | Octal mode string |
| `chown uid:gid file` | `FS.CHOWN vol /file uid gid` | Separate uid and gid args |
| `ln -s target link` | `FS.LN vol /target /link` | Relative or absolute target |
| `readlink link` | `FS.READLINK vol /link` | Returns raw target string |
| `tree dir` | `FS.TREE vol /dir` | Nested array structure |
| `tree -L 2 dir` | `FS.TREE vol /dir DEPTH 2` | Limits recursion depth |
| `du -sh` / `df` | `FS.INFO vol` | File/dir/symlink counts + total bytes |

### Usage examples

**Write and read:**
```bash
redis-cli FS.ECHO vol /hello.txt "Hello, World!"
redis-cli FS.CAT vol /hello.txt
```

**Append:**
```bash
redis-cli FS.ECHO vol /log.txt "first line" APPEND
redis-cli FS.ECHO vol /log.txt "second line" APPEND
```

**Directories:**
```bash
redis-cli FS.MKDIR vol /data/projects PARENTS
redis-cli FS.LS vol /data LONG
```

**Search by filename:**
```bash
redis-cli FS.FIND vol / "*.md"
redis-cli FS.FIND vol /src "*.go" TYPE file
```

**Search file contents:**
```bash
redis-cli FS.GREP vol / "*TODO*"
redis-cli FS.GREP vol /src "*error*" NOCASE
```

**BM25-ranked search (RedisSearch via redis-qmd):**
```bash
redis-qmd --key vol index create
redis-qmd --key vol search "TODO auth"
redis-qmd --key vol query "\"disk full\" AND retry path:/logs/"
redis-qmd --key vol grep "TODO"
```

**Copy, move, delete:**
```bash
redis-cli FS.CP vol /config.json /config.json.bak
redis-cli FS.CP vol /src /src-backup RECURSIVE
redis-cli FS.MV vol /draft.txt /final.txt
redis-cli FS.RM vol /temp RECURSIVE
```

**Symlinks:**
```bash
redis-cli FS.LN vol /config.json /current-config
redis-cli FS.READLINK vol /current-config
```

**Filesystem overview:**
```bash
redis-cli FS.INFO vol
redis-cli FS.TREE vol / DEPTH 2
```

## Bulk Import via redis-cli

If you need to import files directly via `redis-cli` without the `rfs migrate` command:

```bash
cd /path/to/local/dir && find . -type f | while read -r f; do
  redis-cli FS.ECHO vol "/${f#./}" "$(cat "$f")"
done
```

Optional — preserve permissions:
```bash
cd /path/to/local/dir && find . -type f | while read -r f; do
  path="/${f#./}"
  redis-cli FS.CHMOD vol "$path" "0$(stat -c '%a' "$f")"
  redis-cli FS.CHOWN vol "$path" "$(stat -c '%u' "$f")" "$(stat -c '%g' "$f")"
done
```

Or use `rfs migrate <dir>` which handles import, archiving, and mounting in one step. If the source directory contains a `.rfsignore`, matching files and directories are skipped using `.gitignore`-style patterns.

## Volume Management

```bash
redis-cli DEL vol                     # delete entire filesystem
redis-cli EXPIRE vol 3600             # auto-expire after 1 hour
redis-cli SCAN 0 TYPE redis-fs0       # list all filesystem volumes
redis-cli RENAME staging production   # rename a volume
```

## Key Differences and Gotchas

1. **All paths must be absolute** — start with `/`. There is no working directory.
2. **No `cd` or `pwd`** — every command takes the full path explicitly.
3. **Grep uses glob patterns, not regex** — use `*error*` not `.*error.*`.
4. **FS.GREP returns triples** — each match is `[filepath, line_number, line]`.
5. **FS.FIND matches basename only** — `FS.FIND vol / "*.md"` matches `/docs/README.md`.
6. **FS.ECHO auto-creates parents** — no need to `mkdir -p` before writing.
7. **Symlinks resolve at read time** — max 40 levels; cycles produce an error.
8. **Large recursive operations block Redis** — partition across multiple keys for millions of files.
9. **No streaming reads** — `FS.CAT` returns the entire file at once.
10. **Permission bits are metadata only** — `FS.CHMOD`/`FS.CHOWN` store values but Redis does not enforce them.

## Troubleshooting

| Problem | Solution |
|---------|----------|
| `no configuration found` | Run `rfs setup` or create `rfs.config.json` next to the binary |
| `module not loaded` | Load with: `redis-cli MODULE LOAD /path/to/module/fs.so` |
| `cannot find redis-server` | Install Redis, or set `useExistingRedis: true` |
| `cannot find redis-fs-mount` | Run `make mount` in the repo root |
| `mount did not become ready` | Check `mountLog` path for errors |
| `redis-fs is already running` | Run `rfs down` first |

Comments 0

Latest | Hot

Please login before commenting.

Loading comments...

redis-fs

Directory Structure

SKILL.md

Report

Notice