feat: Sync from noir #11294
Merged
feat: Sync from noir #11294
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…oir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
…oir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
…oir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
…oir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
…oir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
…oir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
* master: (442 commits) feat(p2p): request response node sampling (#11330) chore: Trace propagation from json rpc client to server (#11325) feat(p2p): send goodbye messages on disconnecting to peers (#10920) fix(bootstrap): include crates in noir projects hashes (#11344) chore: Add OTEL_EXCLUDE_METRICS (#11317) chore: Test starting multiple anvils allocates distinct ports (#11314) fix: move eslint in circuits.js to dev deps (#11340) fix: update devbox (#11339) chore(p2p): reorganise reqresp handlers (#11327) fix(readme): remove stale link (#11333) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg chore: bump CRS and constants (#11306) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg chore: redo typo PR by nnsW3 (#11322) ...
TomAFrench
approved these changes
Jan 20, 2025
…oir-lang/noir#7120) feat(ssa): Pass to preprocess functions (noir-lang/noir#7072) chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105) fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
…oir#7120) feat(ssa): Pass to preprocess functions (noir-lang/noir#7072) chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105) fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
|
Managed to reproduce this failure on the mainframe as follows: Build the system: ./bootstrap.sh clean
git fetch origin
git checkout --track origin/sync-noir
./bootstrap.sh full
#npm install --global corepack --force
#yarn set version 4.5.2; yarn installIt changed since I last tried and takes a long time, but succeeded. Then we need to build images and run the test that ci runs; luckily there is bootstrap entry point for it, which also takes care of building the necessary docker images: ./bootstrap.sh test-e2e -- e2e_fees_failuresAlternatively the images can be built as follows: ./bootstrap.sh image-e2eThis shows the same failure as on CI. After this when we change something in the Noir code under Now, when we run Unfortunately this doesn't work, complaining that the commit wasn't pushed: Earthly errorTrying again after creating a branch for my investigations: This time it does the build, but it looks like it's running Okay so either I figure out a different way of running the test script directly against a running container, or use this workflow, but it's less likely that I can limit the build if it has to rebuild an EDIT: It's been building the image for over 30 minutes now, at the moment creating the VK for the second protocol contract, which seem to take ages. I used to be able to run these; maybe at the time it ran natively using more CPUs. I'll check it in the morning. EDIT2: After about 10 more minutes the test ran. I can confirm that disabling the |
…g/noir#7047) feat: unconstrained optimizations for BoundedVec (noir-lang/noir#7119) chore: mark libs good (noir-lang/noir#7123) chore: remove comments for time/memory benchmarks (noir-lang/noir#7121) fix: don't always use an exclusive lock in `nargo check` (noir-lang/noir#7120) feat(ssa): Pass to preprocess functions (noir-lang/noir#7072) chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105) fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
feat: unconstrained optimizations for BoundedVec (noir-lang/noir#7119) chore: mark libs good (noir-lang/noir#7123) chore: remove comments for time/memory benchmarks (noir-lang/noir#7121) fix: don't always use an exclusive lock in `nargo check` (noir-lang/noir#7120) feat(ssa): Pass to preprocess functions (noir-lang/noir#7072) chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105) fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100) feat: Parser and formatter support for `enum`s (noir-lang/noir#7110) feat(brillig): SSA globals code gen (noir-lang/noir#7021) feat: `loop` keyword in runtime and comptime code (noir-lang/noir#7096) chore: Add benchmarking dashboard (noir-lang/noir#7068) feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088) feat(ssa): Add flag to DIE pass to be able to keep `store` instructions (noir-lang/noir#7106) chore: Cookbook Onboard integration (noir-lang/noir#7044) chore: lock to ubuntu 22.04 (noir-lang/noir#7098) fix: Remove unused brillig functions (noir-lang/noir#7102) chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
|
Found the way to execute the test without involving the cd yarn-project/end-to-end
test_name=e2e_fees_failures
test_path=$(yq e ".tests.${test_name}.test_path" "scripts/e2e_test_config.yml")
export LOG_LEVEL=debug
yarn test:e2e $test_path -t "fails transaction that error in setup" This runs the failure tests, which uses the FeesTest fixture to set up the test, including the This way we can rebuild after a change with the following commands: ./noir/bootstrap.sh
./noir-projects/bootstrap.sh
./yarn-project/bootstrap.shThese take 5, 10 and 2 minutes, respectively, so ideally we should still narrow it down to the contract that fails. EDIT: |
|
I've disabled the sync workflow so that this branch is stable while we debug. |
|
With Instead we fail to simulate the kernel proof. With some extra logging added: I tried to log the input witness in bb_private_kernel_prover like this: const outputWitness = await this.simulationProvider
.executeProtocolCircuit(witnessMap, compiledCircuit)
.catch((err: Error) => {
this.log.debug(`Failed to simulate ${circuitType}`, {
circuitName: mapProtocolArtifactNameToCircuitName(circuitType),
witness: witnessMap,
error: err,
});
throw err;
});Unfortunately it shows up as empty in the logs, not sure if that is the problem. It's called from the kernel_prover (To iterate these it's enough to run |
|
Logging the input witnesses with |
|
Here it is in JSON as well. It was captured like so: NB the data can be captured during the run if the I'll tweak |
|
After disabling the step marking the arrays in the terminator as mutable in ac9e584, the integration tests fail with the same assertion on a different input (attached). This time re-enabling both the terminator and input parameter marking makes it go away 🙃 Going to try moving part of RC tracking to the function level. |
|
Here's a way to iterate faster on testing #11294 (comment) I assume that a full build has been done before on one's mainframe, and we're in If we change something in ./noir/bootstrap.shIf we change something in ./noir-projects/noir-contracts/bootstrap.sh compile stateful_test_contract && \
cp ./noir-projects/noir-contracts/target/stateful_test_contract-StatefulTest.json \
./yarn-project/noir-contracts.js/artifacts/stateful_test_contract-StatefulTest.json Finally execute the test: cd yarn-project/simulator && yarn test -t "should run the destroy_and_create function" |
|
Here's what I managed to find out during debugging the The test fails because it expects to find 2 non-empty nullifiers (the array is 16 long). An empty nullifier is where note hash, the nullifier, and the counter are all zero, like this: The contract and the libraries do emit notifications about nullifiers, which I am able to see if I add some extra logging in the I can add an assertion in the and the circuit will complete, only the test will fail because it will not find any nullifiers, so there are nullifiers in there. If on top of that I add the following assertion: then the test passes, it finally gets the nullifiers in the return value. So I can't see what the NB I did these experiments with the following things disabled:
We can have a look at the SSA of the method under test with this: |
|
I'm not sure yet why, but things seem to go south after the 2nd Mem2Reg pass. Before that, the return value from After it, most of the return value is 0, except a few fields at the end which seem to be coming from the inputs Adding back the assertions mentioned above preserves the variables to be returned: |
I noticed now that I forgot to disable one of the new passes, namely the Mem2Reg (1st) pass which was added after the new Inlining Simple Functions pass. Testing indicates that removing just this SSA pass fixes the test. |
* master: (67 commits) chore: pull out more sync stuff (#11617) refactor: consolidate contract functions in pxe (#10480) feat: txe native world state (#11226) feat(avm): vm2 bitwise subtrace (#11473) fix: boxes playwright + web bundles (#11615) chore: pull lazy memory initialization from sync (#11607) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg chore: add method to filemanager (#11610) feat(avm): get_row optimization - 25x faster logderiv inv (#11605) fix(spartan): get svc addresses from load balancers (#11606) chore: remove old public circuit public inputs (#11594) fix: fix boxes (#11592) fix: only set node version in node pools (#11604) chore: replace usage of stdlib with sha256 library (#11394) fix: pin k8s version (#11599) chore: make target contract public (#11582) fix(spartan): reduce l1 deploy retry time (#11591) ...
|
The latest build failure % cargo run -q -p nargo_cli -- --program-dir ../../noir-projects/noir-contracts compile --package token_contract --silence-warnings --inliner-aggressiveness 0 35s ~/aztec-packages/noir/noir-repo sync-noir+ akosh-box
The application panicked (crashed).
Message: Attempted to insert instruction not handled by runtime: EnableSideEffectsIf { condition: Id(211) }
Location: compiler/noirc_evaluator/src/ssa/ir/dfg.rs:306
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
...
noirc_evaluator::ssa::ssa_gen::program::Ssa>::inline_functions_with_no_predicates::h7e950d60a7dac5a8
at /mnt/user-data/akosh/aztec-packages/noir/noir-repo/compiler/noirc_evaluator/src/ssa/opt/inlining.rs:56
36: noirc_evaluator::ssa::optimize_all::{{closure}}::h24a6951eec9d0574
at /mnt/user-data/akosh/aztec-packages/noir/noir-repo/compiler/noirc_evaluator/src/ssa.rs:190It happens in the Inlining (2nd) pass, and only with the non-default |
|
The latest test failure is: % cd yarn-project/bb-prover && yarn test -t "public fee payment" ; cd -
FAIL src/avm_proving_tests/avm_proving.test.ts (5.11 s)
● AVM WitGen & Circuit › public fee payment › fee payment
Not enough balance for fee payer to pay for transaction (got 10000 needs 16794)
435 | if (currentBalance.lt(txFee)) {
436 | if (this.enforceFeePayment) {
> 437 | throw new Error(
| ^
438 | `Not enough balance for fee payer to pay for transaction (got ${currentBalance.toBigInt()} needs ${txFee.toBigInt()})`,
439 | );
440 | } else {
at PublicTxSimulator.payFee (../../simulator/src/public/public_tx_simulator.ts:437:15)
at PublicTxSimulator.simulate (../../simulator/src/public/public_tx_simulator.ts:118:5)
at AvmProvingTester.simulateTx (../../simulator/src/public/fixtures/public_tx_simulation_tester.ts:122:23)
at AvmProvingTester.simProveVerify (avm_proving_tests/avm_proving_tester.ts:89:20)
at Object.<anonymous> (avm_proving_tests/avm_proving.test.ts:424:9)
Test Suites: 1 failed, 2 skipped, 1 of 3 total
Tests: 1 failed, 21 skipped, 22 total
Snapshots: 0 total
Time: 5.521 s
Ran all test suites with tests matching "public fee payment".It looks like the call consumed more gas than expected. The test account gets a balance of 10000 hardcoded in the test, but the 16794 is based on the gas consumed in the transaction (which is just adding two numbers): |
|
After increasing the initial balance to 20000 we have this (with What's strange is that x4608 does equal 17928, x4e20 is 20000, so why does barretenberg fail the test like this? |
Hmm that is very strange. In if (current_balance < tx_fee) {
info("Not enough balance for fee payer to pay for transaction (got ", current_balance, " needs ", tx_fee);
throw std::runtime_error("Not enough balance for fee payer to pay for transaction");
}So we are failing if the |
aakoshh
requested review from
jeanmon,
IlyasRidhuan and
fcarreiro
as code owners
Changes to public function bytecode sizes
🧾 Summary (100% most significant diffs)
Full diff report 👇
|
Changes to circuit sizes
🧾 Summary (100% most significant diffs)
Full diff report 👇
|
|
For posterity: the docs of the Also: the |
TomAFrench
reviewed
Jan 31, 2025
Comment on lines
+163
to
164
| && self.structs.is_empty() | ||
| && self.type_aliases.is_empty() |
There was a problem hiding this comment.
@jfecher looks like this didn't get updated to reflect enums
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Automated pull of development from the noir programming language, a dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
chore: let
add_definition_locationtake a Location (noir-lang/noir#7185)fix(LSP): correct signature for assert and assert_eq (noir-lang/noir#7184)
chore(experimental): Prevent enum panics by returning Options where possible instead of panicking (noir-lang/noir#7180)
feat(experimental): Construct enum variants in expressions (noir-lang/noir#7174)
feat: add
noir-inspector(noir-lang/noir#7136)fix: ensure canonical bits decomposition (noir-lang/noir#7168)
fix: Keep
inc_rcfor array inputs during preprocessing (noir-lang/noir#7163)fix(docs): Update broken links to EC lib (noir-lang/noir#7141)
feat: inline simple functions (noir-lang/noir#7160)
feat(ssa): Expand feature set of the Brillig constraint check (noir-lang/noir#7060)
fix(ssa): Resolve value before fetching from DFG in a couple cases (noir-lang/noir#7169)
fix:
Function::is_no_predicatesalways returned false for brillig f… (noir-lang/noir#7167)chore(refactor): Remove globals field on Ssa object and use only the shared globals graph (noir-lang/noir#7156)
chore: let
Function::inlinedtake ashould_inline_callfunction (noir-lang/noir#7149)chore: add compile-time assertions on generic arguments of stdlib functions (noir-lang/noir#6981)
fix: LSP hover over function with
&mut self(noir-lang/noir#7155)feat(brillig): Set global memory size at program compile time (noir-lang/noir#7151)
feat: LSP autocomplete module declaration (noir-lang/noir#7154)
feat(ssa): Reuse constants from the globals graph when making constants in a function DFG (noir-lang/noir#7153)
feat: LSP chain inlay hints (noir-lang/noir#7152)
chore: turn on overflow checks in CI rust tests (noir-lang/noir#7145)
fix(ssa): Use post order when mapping instructions in loop invariant pass (noir-lang/noir#7140)
fix: preserve types when reading from calldata arrays (noir-lang/noir#7144)
feat: Resolve enums & prepare type system (noir-lang/noir#7115)
feat:
loopmust have at least onebreak(noir-lang/noir#7126)feat: parse globals in SSA parser (noir-lang/noir#7112)
fix: allow calling trait impl method from struct if multiple impls exist (noir-lang/noir#7124)
fix: avoid creating unnecessary memory blocks (noir-lang/noir#7114)
chore: relax threshold for reporting regressions (noir-lang/noir#7130)
fix: proper cleanup when breaking from comptime loop on error (noir-lang/noir#7125)
fix: Prevent overlapping associated types impls (noir-lang/noir#7047)
feat: unconstrained optimizations for BoundedVec (noir-lang/noir#7119)
chore: mark libs good (noir-lang/noir#7123)
chore: remove comments for time/memory benchmarks (noir-lang/noir#7121)
fix: don't always use an exclusive lock in
nargo check(noir-lang/noir#7120)feat(ssa): Pass to preprocess functions (noir-lang/noir#7072)
chore: Formatting issues / minor errors in the docs (noir-lang/noir#7105)
fix: defunctionalize pass on the caller runtime to apply (noir-lang/noir#7100)
feat: Parser and formatter support for
enums (noir-lang/noir#7110)feat(brillig): SSA globals code gen (noir-lang/noir#7021)
feat:
loopkeyword in runtime and comptime code (noir-lang/noir#7096)chore: Add benchmarking dashboard (noir-lang/noir#7068)
feat(experimental): try to infer lambda argument types inside calls (noir-lang/noir#7088)
feat(ssa): Add flag to DIE pass to be able to keep
storeinstructions (noir-lang/noir#7106)chore: Cookbook Onboard integration (noir-lang/noir#7044)
chore: lock to ubuntu 22.04 (noir-lang/noir#7098)
fix: Remove unused brillig functions (noir-lang/noir#7102)
chore(ssa): Use correct prefix when printing array values in global space (noir-lang/noir#7095)
END_COMMIT_OVERRIDE