- You
- Agents
Tell your agent what the firmware should do. It writes the suite, runs it on the virtual board, and returns a pass or fail for each check. The same checks can fail a pull request. Setup is on Test in CI.
When to use it
Use a test when each check needs a result. A firmware that stops printing fails the test. When you want the printed output, use run.Ask your agent
This build is fixed at 35°C. Above 30°C it should log a blink every 3 seconds. Write a test that it boots, reaches that blink without a fault, and logs the blink within 3 seconds on the STM32F4 Discovery, and run it.
Holding the user button for 2 seconds should switch modes and turn the user LED on. Write a test for it on the STM32F4 Discovery and run it.
What you get back
You get a pass or fail for each test.succeeded means the suite ran, not that every test passed. Check result.summary.What you can test
A suite advances virtual time, watches UART lines, and drives the board: it presses buttons, reads LEDs, and sends input over UART.| Scenario | What the suite does | Good to know |
|---|---|---|
| Boot | Expects a ready line within a few seconds. | |
| Init order | Expects several lines in order. | Each check only sees output printed after it starts. |
| Timing | Checks a one-second tick against chip-accurate interrupt timing. | It reports matched or timed_out, not how long the interval actually was. |
| Long-run liveness | Runs the firmware for minutes, then expects a line that shows it is still alive. | Time advances in chunks of up to 60 seconds. |
| HardFault | Expects a line the firmware prints after the code that could fault. | A fault is not reported on its own: the line never arrives and the check fails. A handler that prints the fault address makes the stop visible on the UART. |
| Buttons | Presses, holds, or taps the board’s buttons at an exact point in virtual time. | A 2 second hold is exactly 2 seconds on every run. |
| LEDs | Reads whether an LED is on or off. | The firmware does not need to log it. |
| Commands over UART | Types a command into a firmware shell, or sends raw bytes for a binary protocol, then expects the reply. | Write after the firmware’s ready line, or the bytes may be lost. |
Not supported yet
A suite can use the buttons and LEDs the board already has. Extra buttons or LEDs wired to other pins, other pin-state or register and memory reads, and external sensors or I2C/SPI devices are not supported yet.To test a sensor reading such as a temperature, build a fixed value into the firmware, then expect the line it prints. The suite does not feed the value in.run_test_suite
run_test_suite takes:The TypeScript suite file, from a second
issue_upload_ticket. Ids look like artifact_....The virtual board.
model is a board model such as stm32f4_discovery. uarts is the list of UART names to capture, for example ["usart2"].issue_upload_ticket twice, both with the same project_id. It PUTs the ELF to the first upload_url and the suite file to the second. run_test_suite returns run_test_suite_id as soon as the job starts. Pass that id to wait_for_job as job_id.How it works
The suite drives the simulation. The board is paused until the suite advances time. Virtual time moves only inside the awaited calls. Between those calls the board stays frozen.Only the UARTs named inboard.uarts are watched by runUntilUartLine and available to the suite.Writing a suite
A suite is one TypeScript file:import { expect } from "jsr:@std/expect";
Deno.test("firmware greets", async () => {
const outcome = await simulation.runUntilUartLine("Hello world!", 10000);
expect(outcome.kind).toBe("matched");
});
Deno.test, or describe and it from jsr:@std/testing/bdd. The only imports available are jsr:@std/testing/bdd, jsr:@std/expect, and zod. No network, npm packages, or imports of other files. Chiplab type-checks the suite before it runs.The server provides the simulation and fireEvent types itself. The suite must not reference chiplab.d.ts. Only the suite file is uploaded, and it is checked without network access, so a /// <reference> to the definitions, by local path or URL, fails to resolve and the suite does not compile.For editor support, point the project’s deno.json at the published definitions:{ "compilerOptions": { "types": ["https://chiplab.veecle.ai/mcp/resources/schemas/chiplab.d.ts"] } }
deno check then needs --allow-import to fetch them.That URL always serves the latest definitions. Its @version tag says which version they are. To pin one, use https://chiplab.veecle.ai/schemas/chiplab/v<version>/chiplab.d.ts, whose content never changes.The simulation API
The suite gets two globals,simulation and fireEvent.simulation.runForMillis(millis)runs formillisof virtual time, then pauses.simulation.runUntilUartLine(expected, maxMillis)runs until a line containingexpectedappears on any UART inboard.uarts, ormaxMilliselapses. It resolves to{ kind: "matched" }or{ kind: "timed_out" }.simulation.runUntilUartLineOn(uart, expected, maxMillis)does the same, but only watchesuart.
runUntilUartLineandrunUntilUartLineOnonly match output produced after the call starts.expectedmust not contain control characters such as a newline.
Peripherals
Look up a peripheral onsimulation, then drive it with fireEvent.simulation.getButton(name?),simulation.getLed(name?), andsimulation.getUart(name?)return one peripheral.simulation.getAllUarts()returns every UART inboard.uarts.- Button and LED names come from the board, for example
UserButtonandUserLEDonstm32f4_discovery. A UART is named by its id, for exampleusart2. - Without a
name, a lookup succeeds only if the board has exactly one peripheral of that kind. If nothing matches, or several match and nonamepicks one, the lookup throws and the message lists the available names.
fireEvent.press(button)presses a button and holds it down.fireEvent.release(button)lets it go.fireEvent.click(button)presses and releases at the same instant. Firmware that polls the input level never sees it. Only edge-triggered code, such as a GPIO interrupt handler, does. To hold a button across virtual time, callpress, advance time, then callrelease.led.state()resolves to"on"or"off", as of the last pause.fireEvent.write(uart, data)sendsdatato the firmware: a string, sent as UTF-8, such as"help\r\n", or aUint8Arrayfor binary protocols. Write after the firmware has set up the UART, for example after its ready line. Bytes sent earlier may be lost.
Example checks
Boot. ExpectSystem ready within 3 seconds.Deno.test("boots", async () => {
const outcome = await simulation.runUntilUartLine("System ready", 3000);
expect(outcome.kind).toBe("matched");
});
Deno.test("init order", async () => {
expect((await simulation.runUntilUartLine("clocks on", 3000)).kind).toBe("matched");
expect((await simulation.runUntilUartLine("uart on", 3000)).kind).toBe("matched");
expect((await simulation.runUntilUartLine("System ready", 3000)).kind).toBe("matched");
});
tick 5 within 5.5 seconds. That relies on the chip-accurate interrupt timing. The call resolves to matched or timed_out. It does not report how long the interval actually was.Deno.test("one second tick", async () => {
const outcome = await simulation.runUntilUartLine("tick 5", 5500);
expect(outcome.kind).toBe("matched");
});
runForMillis takes at most 60000 ms, so a longer run is 60 second chunks. Then expect alive. The line has to be printed after those chunks, because the next call only sees output from that point on.Deno.test("stays alive", async () => {
await simulation.runForMillis(60000);
await simulation.runForMillis(60000);
const outcome = await simulation.runUntilUartLine("alive", 3000);
expect(outcome.kind).toBe("matched");
});
Deno.test("stubbed temperature", async () => {
const outcome = await simulation.runUntilUartLine("Temp: 23.5C", 3000);
expect(outcome.kind).toBe("matched");
});
UserButton for 2 seconds should turn UserLED on. A click would not work here if the firmware polls the button.Deno.test("long press turns the LED on", async () => {
expect((await simulation.runUntilUartLine("System ready", 3000)).kind).toBe("matched");
const button = await simulation.getButton("UserButton");
const led = await simulation.getLed("UserLED");
await fireEvent.press(button);
await simulation.runForMillis(2000);
await fireEvent.release(button);
await simulation.runForMillis(100);
expect(await led.state()).toBe("on");
});
help, and expect the reply on the same UART.Deno.test("help lists commands", async () => {
const uart = await simulation.getUart("usart2");
expect((await simulation.runUntilUartLineOn(uart, "System ready", 3000)).kind).toBe("matched");
await fireEvent.write(uart, "help\r\n");
expect((await simulation.runUntilUartLineOn(uart, "Commands:", 1000)).kind).toBe("matched");
});
Reading the result
succeeded means the suite ran, not that it passed. Check result.summary.| What happened | wait_for_job |
|---|---|
| The suite ran | status is succeeded. result.summary has planned, passed, and failed. |
| The suite did not compile, or it threw before any test ran | status is succeeded. result.summary is null. Read testsuite_stderr. |
| The suite ran past its time limit | status is failed. error is test suite timed out after 300s, with no result. See Failed runs. |
| The simulation failed and the run was aborted | status is failed. error is internal server error, with no result. See Failed runs. |
Hello world! on stm32f4_discovery / usart2, run against firmware that prints that line:{
"input": {
"board": { "model": "stm32f4_discovery", "uarts": ["usart2"] },
"firmware_artifact_id": "artifact_01m3ye77kfec79agpsjamg34nf",
"suite_artifact_id": "artifact_01m3ye76e2fkrb99wrpwvae4xa"
},
"job_id": "run_test_suite_01m3ye78dkf5na6977req4trz6",
"project_id": "project_01m3ydfkccerpswp3zh14rpeqh",
"result": {
"artifacts": ["testsuite_report", "testsuite_stdout", "firmware_uart"],
"summary": { "failed": 0, "passed": 1, "planned": 1 }
},
"status": "succeeded",
"timed_out": false
}
before fault and then hits a HardFault. The wait times out, the assertion fails, and the job still succeeds:{
"input": {
"board": { "model": "stm32f4_discovery", "uarts": ["usart2"] },
"firmware_artifact_id": "artifact_01m3ye7hgaeyya3vwmaye3ep9g",
"suite_artifact_id": "artifact_01m3ye76e2fkrb99wrpwvae4xa"
},
"job_id": "run_test_suite_01m3ye7j9bfjhsv9spkmepxa5v",
"project_id": "project_01m3ydfkccerpswp3zh14rpeqh",
"result": {
"artifacts": ["testsuite_report", "testsuite_stdout", "testsuite_stderr", "firmware_uart"],
"summary": { "failed": 1, "passed": 0, "planned": 1 }
},
"status": "succeeded",
"timed_out": false
}
Artifacts
result.artifacts lists what issue_download_ticket can download. Pass the run_test_suite_id as job_id.testsuite_report: the test runner’s own report, stored verbatim. Its format is not guaranteed, so read the counts fromresult.summary.testsuite_stdoutandtestsuite_stderr: the suite’s output, includingconsole.log.firmware_uart: the firmware’s UART output. A run calls this artifactuart.simulator_stdoutandsimulator_stderr: simulator logs.
testsuite_stderr.Failed runs
A failed job has noresult, so its artifacts are not listed. The output captured up to the failure is usually still stored. Request the artifacts above by name with issue_download_ticket. A ticket is issued for any name, and fetching one that was not stored fails.Once the simulation has failed, the suite’s remaining simulation calls throw. Look in testsuite_stderr and simulator_stderr first for the original error.Limits
The whole suite has 5 minutes of wall-clock time. Exceeding it fails the job withtest suite timed out after 300s. See Failed runs.Each runForMillis, runUntilUartLine, and runUntilUartLineOn call takes 1 to 60000 ms.Each fireEvent.write call takes up to 1024 bytes of UTF-8.Steps (t.step) and nested it calls count as tests of their own, named parent > step, alongside the test that contains them.