|
PineForge v0.13.1-379-g9b50973
Deterministic PineScript v6 backtest runtime — C ABI reference
|
Thanks for your interest. This document is the whole workflow, written for someone who has never seen this repository. Read it once before your first change; after that, the checklist near the end is the part you keep.
Community interaction follows the Code of Conduct. For licensing and third-party obligations (Eigen, optional benchmark tools), see LEGAL.md. If you are an AI agent working from a brief rather than a person reading a guide, read Contributing as an LLM instead — same ground, written as rules with the guard that enforces each one.
PineForge is a C++17 backtest and forward-execution engine with a C ABI. Its job is to be right: a strategy run through it produces the same trade list TradingView produces for the same PineScript on the same bars, trade for trade, and two runs of the same inputs produce identical bytes. The PineScript → C++ transpiler is a separate project (pineforge-codegen); this repository is the runtime that every compiled strategy links against, and also a kernel that runs strategies written directly in C++ or C with no PineScript anywhere.
The rule: the kernel must not know what TradingView is. The test is mechanical — if justifying the change needs the word "TradingView", it does not belong in the kernel. The kernel changes only for a generic capability with a recorded ruling; TradingView's own rules go in the adapter or in codegen. The boundary, its five other rules and the ruling table live in ADR 0001.
Two consequences that are easy to miss:
NativeRunSpec field adds its row to ADR 0001's ruling table: native-only (and then it needs a native example and a test), adapter-policy, or adapter-hook. scripts/check_native_feature_rulings.py fails until the table is true.| You want to… | It goes in | And you must also |
|---|---|---|
| Fix a TradingView-parity difference | src/source/ or src/compat/pine/ | add a replay test on recorded bars; run the parity gate |
| Add a generic broker/matching capability | the kernel (src/engine_*, src/native_*) | make it opt-in, add its ADR 0001 ruling row, add a kernel-only test, and add an examples/native/ host if the ruling is native-only |
| Add or change a TA class | the right ta_*.cpp partition + its declaration in <pineforge/ta.hpp> | add a unit test against a hand-computed series |
| Change what codegen may emit | the contract, not this repo's runtime | say so in the PR; the transpiler lives in pineforge-codegen-oss |
Add a runtime PF_API export | src/c_abi.cpp + include/pineforge/pineforge.h | update EXPECTED_RUNTIME check_c_abi_runtime.py:28, the ctypes harnesses, and the README symbol table — all in the same commit |
| Add a C kernel-driving export | src/native_c_host.cpp + include/pineforge/native_c_api.h | update that header's COVERAGE block; scripts/check_native_c_api_surface.py proves it is exactly the host's public surface |
| Document something | docs/pages/, README.md, this file | cite the tree by file:line; the anchor guard checks that the line still holds the symbol |
You need CMake ≥ 3.16, a C++17 compiler (GCC ≥ 9, Clang ≥ 10, Apple Clang ≥ 12), Python 3 and Eigen 3.3+ (fetched automatically if absent).
The stale-test-binary trap.
cmake --build build --target pineforgerebuilds only the static library. Test executables are separate targets that statically link it, andctestrebuilds nothing — so after a library-only build,ctestruns stale binaries and can pass falsely. Always build all targets beforectestwhen runtime values changed.
A change is proven by a test that failed before it and passes after it. In that order, deliberately:
tests/, or a native host under examples/native/, or a corpus probe. Register it: TEST_SOURCES tests/CMakeLists.txt:1 for a unit, or PINEFORGE_NATIVE_EXAMPLES examples/native/CMakeLists.txt:16 plus a _pf_example_line_hello_kernel examples/native/CMakeLists.txt:98 for the row's own assertion.A test whose body is a seed row, a return 0 main, a disabled row or a TODO placeholder is not evidence. Neither is a test whose expected value you copied out of the run you are trying to justify.
Run these before you push. Each one refuses a specific way of being wrong.
What each gate refuses:
| Gate | Refuses |
|---|---|
check_c_abi_runtime.py | a PF_API runtime export added or removed without its inventory row |
check_native_c_api_surface.py | a public NativeStrategyHost member with no C spelling and no recorded reason |
check_native_feature_rulings.py | a NativeRunSpec field the adapter does not declare and the ADR does not rule |
check_kernel_residuals.py | a TradingView-shaped name reaching the kernel archive without an ADR 0001 row |
check_native_cpp_versions.py, check_aggregate_cpp_versions.py | an internal C++ epoch moved without its consumers |
check_adapter_spec_shadowing.py | the adapter setting a kernel field it is ruled not to set |
check_twin_parity.py | a frozen test assertion rewritten instead of a behaviour change being argued |
check_doc_anchors.py | a file:line citation that no longer points at the symbol it claims |
| check_doc_lint.py | a stale epoch, a roadmap label or a "there is no … yet" claim the tree has falsified | | check_pine_to_native_coverage.py | a Pine builtin with no row on the migration page | | the CTest row floors | a test row that vanished from a profile |
ci_verify.py counts the CTest rows that actually ran and fails below a floor — KERNEL_MIN_TESTS ci_verify.py:93 and RELEASE_MIN_TESTS ci_verify.py:116. A deleted or silently skipped row is a failure, not a quieter run. If your change adds rows, raise the floor in the same commit and say by how much; if it legitimately removes one, lower it deliberately and say why. --min-tests overrides the floor for a local experiment; never in a commit.
The validation corpus is a byte oracle, not a smoke test.
It checks the corpus out at the gitlink this repository records, refuses to start if a pinned input under validation/ is modified, runs the probes, and fails if any probe's engine_trades.csv no longer hashes to what scripts/corpus_parity_baseline.txt pins — naming the probe and printing the first differing rows — or if scripts/verify_corpus.py no longer prints its pinned tier headline.
"Moved" means any byte changed: a one-tick slippage difference moves a probe's baseline hash while the tier headline stays excellent=311. Both matter. The baseline, not the corpus's committed trades, is the oracle: the tapes pineforge-corpus committed were written by an older harness and no longer reproduce.
A refresh is deliberate and carries its evidence in the same commit:
The subset runs as a required check on a pull request; the full sweep runs nightly and on demand. The details, including which 30 probes and why, are in docs/ci.md.
Sometimes a pinned number should change — a test's expected value, a floor, a baseline hash. Loosening a pin to make a red thing green is the single easiest way to destroy this repository's value, so the convention is explicit: the diff that changes a pin carries, on the line above it or in the commit message,
with the what naming the mechanism, not the symptom. No note, no merge. You will find the convention already in scripts/ci_verify.py and in the test suites; follow the spelling exactly so it stays greppable.
Two different promises:
<pineforge/pineforge.h>, <pineforge/native_c_api.h>) is append-only within a major version: fields and functions are added at the end, never reordered, removed or retyped. PATCH never touches it, MINOR appends, MAJOR breaks and needs maintainer signoff. The static_asserts in src/c_abi.cpp pin the layouts. If those asserts fail in your branch, do not "fix" them by changing the asserts — find what changed in the C++ representation.inline namespace engine_script_run_v… and the hash domains) guard link-time compatibility between the library and the strategies compiled against it. Moving one is a real event with its own procedure: read docs/pages/abi-stability.md before you do, and expect the version guards to fail until every consumer moves with you.std::filesystem, no <format>. Yes to structured bindings, if constexpr, std::optional..clang-format is canonical.lower_snake_case functions and members, PascalCase types.BacktestEngine method goes in the engine_*.cpp partition that matches its concern, a file-local helper into an anonymous namespace, a genuinely cross-TU helper into engine_internal.hpp under pineforge::internal.printf left behind, no commented-out experiment, no test.skip.The coverage harness is opt-in — instrumentation slows the build and leaves profile side-files:
Outputs land in build-cov/coverage/: totals.txt (per-file percentages), uncovered.txt (same rows, lowest first) and per-file/<source>.txt (annotated listings). Honoured env vars: BUILD_DIR, COMPILER, JOBS, SKIP_BUILD=1, SKIP_TESTS=1, FORMAT=html. A PR touching a file under 80 % should not lower that file's line coverage.
Read git log --format=s -40 before your first commit; the convention is visible and consistent:
<Area> is Docs, Tests, Kernel, Examples, Rulings, Integration, or the subsystem. The summary names the mechanism, not the symptom — "the CTest
row floor counts rows that ran; skipped rows are listed, not counted" rather than "fix floor". The body explains why, with the measurement. One concept per pull request; several commits inside it is fine.
A pull request body has two parts and no third:
ci_verify summary lines for both profiles, the parity result, the floor numbers if they moved. Paste it; do not summarise it.All CI must pass before merge: build on Ubuntu + macOS in Release and Debug, sanitizers, ctest, the source guards, the install/find_package smoke test, and the parity subset.
For changes measured against the closed TradingView test set, a sweep is run before and after over a fixed population, and the merge rule is stated in terms of movement between grade bands: no individual regression, and net movement ≥ 0 across the target excellent and excellent+strong bands. A documented native-correctness exception permits exactly net 0 with no individual regression, after full comparison, independent review and green CI; its FAIL stays on the record and baseline promotion is deferred. Negative movement is outside the exception — it is not traded against anything.
The most valuable thing you can send is a reproduction: the Pine script, the OHLCV slice, and TradingView's own trade list exported at full precision. That is exactly how every rule in this engine was found.
By contributing, you agree your contributions will be licensed under the Apache License 2.0 (the same license as the rest of the project). See [LICENSE](LICENSE).
corpus/ and benchmarks/assets to the intended commits and verify both upstream tags resolve under their published Apache-2.0 trees. LEGAL.md describes the submodule split..env, or machine-specific paths in tracked files; keep benchmarks/_workdir, .venv and node_modules untracked.libpineforge (e.g. Eigen). Update LEGAL.md if you add a new mandatory runtime dependency.benchmarks/ tooling installs AGPL-covered PineTS. Default CI stays on ctest only so a minimal clone is not forced to pull AGPL into the library build.