We gave Hearth 20 everyday agent jobs (files, notes, the shell, web pages, memory) and ran each one three times on free local models. A script checks the files each run actually made, so a confident wrong answer counts as a fail.
18 of 20 in every run. All 7 simple jobs every time; the misses were adding up a list of numbers, reading one field from a web page, copying a file in capitals and sorting a table.
Between 12 and 14 of 20 per run. Fine for simple jobs; it never finished the CSV, table-sorting or inbox-summary tasks.
● passed, ○ failed, one dot per run. Each task starts in a fresh workspace with fresh memory. When a task fails locally, Hearth can hand it to a cloud model, but only if you switch that on; it was off for this test.
| # | Task | Level | Qwen3 8B | Qwen2.5 7B |
|---|---|---|---|---|
| 1 | Create a file hello.txt containing exactly: hello world | simple | ●●● | ●●● |
| 2 | What's the wifi password? It's in my notes. | simple | ●●● | ●○○ |
| 3 | How many files are in my workspace? | simple | ●●● | ●●● |
| 4 | Remember that my dog is called Biscuit. | simple | ●●● | ●●● |
| 5 | Use the shell to print the current year with `date +%Y` and tell me what it says. | simple | ●●● | ●●● |
| 6 | What is the page title of https://example.com ? | simple | ●●● | ●●● |
| 7 | Make a shopping list in list.txt with eggs, milk and bread, one per line. | simple | ●●● | ●●● |
| 8 | Add up the numbers in numbers.txt and write just the total to total.txt. | medium | ●○○ | ●○○ |
| 9 | Make open.txt listing only the unfinished tasks from todo.txt, one per line. | medium | ●●● | ●○○ |
| 10 | Fetch https://httpbin.org/json and save the slideshow's title into title.txt. | medium | ●●○ | ●○○ |
| 11 | Copy draft.txt to final.txt with all the text in uppercase. | medium | ●●○ | ●●○ |
| 12 | Use the shell to count the lines in log.txt and tell me the number. | medium | ●●● | ●●● |
| 13 | Which product sold the most units in sales.csv? Write just its name to best.txt. | medium | ●●● | ○○○ |
| 14 | Combine all the files in the notes folder into combined.txt. | medium | ●●● | ●●● |
| 15 | Write a Python script fib.py that prints the first 10 Fibonacci numbers, one per line | complex | ●●● | ●●● |
| 16 | From people.json, write a markdown table of the people older than 30, sorted by age (youngest first), to older.md. | complex | ●○○ | ○○○ |
| 17 | Read every email in the inbox folder and write summary.txt with one line per email: sender - subject. | complex | ●●● | ○○○ |
| 18 | Create a folder called demo with README.md (first line '# Demo') and main.py that prints hi. Then run main.py with python3 and tell me what it printed. | complex | ●●● | ●●● |
| 19 | In config.ini change the server port to 9090. Don't change anything else. | complex | ●●● | ●●○ |
| 20 | Remember that my birthday is 14 March, then write reminders.txt containing: Birthday: 14 March | complex | ●●● | ●●○ |
Setup: NVIDIA RTX 3090 Ti, models served by Ollama, Hearth's default settings (up to 8 tool calls per job). Median time per job: 0.8 s on Qwen3 8B, 0.6 s on Qwen2.5 7B. Shell commands were approved automatically, standing in for you. Gemma 4 12B and Qwen3.8 27B are next on this page.
An 8B model fits on one gaming GPU and handled 90% of these jobs, with nothing leaving the machine and no per-token cost. That's what Hearth is built on. See what your agent costs in the cloud, or check your OpenClaw setup.