Merge v1.14.10

params: begin v1.14.10 release cycle (#30457)

genesis: fix dev mode alloc (#30460)

Balance being null causes `getGenesisState` to fail as the balance field
is required in json marshaling of an account.

core: minor fix for the log wrapper with debug purpose (#30454)

After this PR, ethereum/go-ethereum#28187, the
way to set the default logger is different. This PR only updates the way
to set logger in some test cases' comments that existed in the codebase
(since this commit
ethereum/go-ethereum@b63e3c37a6). Although I
am not sure if it a good way to leave the code in the comment, it truly
makes me more efficiently to debug and fix the failing test cases.

ethdb/pebble: handle errors (#30367)

.github: add release maintainers to params/ CODEOWNERS (#30458)

build: fix macos builds by working around travis osx flaw (#30479)

This should fix ethereum/go-ethereum#30471.
See investigation in ethereum/go-ethereum#30478
for more background.

beacon, core, eth, miner: integrate witnesses into production Geth (#30069)

This PR integrates witness-enabled block production, witness-creating
payload execution and stateless cross-validation into the `engine` API.
The purpose of the PR is to enable the following use-cases (for API
details, please see next section):

- Cross validating locally created blocks:
- Call `forkchoiceUpdatedWithWitness` instead of `forkchoiceUpdated` to
trigger witness creation too.
- Call `getPayload` as before to retrieve the new block and also the
above created witness.
- Call `executeStatelessPayload` against another client to
cross-validate the block.

- Cross validating locally processed blocks:
- Call `newPayloadWithWitness` instead of `newPayload` to trigger
witness creation too.
- Call `executeStatelessPayload` against another client to
cross-validate the block.

- Block production for stateless clients (local or MEV builders):
- Call `forkchoiceUpdatedWithWitness` instead of `forkchoiceUpdated` to
trigger witness creation too.
- Call `getPayload` as before to retrieve the new block and also the
above created witness.
- Propagate witnesses across the consensus libp2p network for stateless
Ethereum.

- Stateless validator validation:
- Call `executeStatelessPayload` with the propagated witness to
statelessly validate the block.

*Note, the various `WithWitness` methods could also *just be* an
additional boolean flag on the base methods, but this PR wanted to keep
the methods separate until a final consensus is reached on how to
integrate in production.*

---

The following `engine` API types are introduced:

```go
// StatelessPayloadStatusV1 is the result of a stateless payload execution.
type StatelessPayloadStatusV1 struct {
	Status          string      `json:"status"`
	StateRoot       common.Hash `json:"stateRoot"`
	ReceiptsRoot    common.Hash `json:"receiptsRoot"`
	ValidationError *string     `json:"validationError"`
}
```

- Add `forkchoiceUpdatedWithWitnessV1,2,3` with same params and returns
as `forkchoiceUpdatedV1,2,3`, but triggering a stateless witness
building if block production is requested.
- Extend `getPayloadV2,3` to return `executionPayloadEnvelope` with an
additional `witness` field of type `bytes` iff created via
`forkchoiceUpdatedWithWitnessV2,3`.
- Add `newPayloadWithWitnessV1,2,3,4` with same params and returns as
`newPayloadV1,2,3,4`, but triggering a stateless witness creation during
payload execution to allow cross validating it.
- Extend `payloadStatusV1` with a `witness` field of type `bytes` if
returned by `newPayloadWithWitnessV1,2,3,4`.
- Add `executeStatelessPayloadV1,2,3,4` with same base params as
`newPayloadV1,2,3,4` and one more additional param (`witness`) of type
`bytes`. The method returns `statelessPayloadStatusV1`, which mirrors
`payloadStatusV1` but replaces `latestValidHash` with `stateRoot` and
`receiptRoot`.

travis: work around travis/osx/go1.23 setup bug (#30491)

This is a work-around for a strange issue with travis, specifically,
`os=osx, go: 1.23.1`. When this is used, the actual go that ends up
being used is `go1.19.4 darwin/amd64 `.

Using `which go`, it told me that the `go` in the path was a softlink at
`/Users/travis/gopath/bin/go1.23.1 `. However, this was not true: using
`command -v go`, it told me that the actual `go` that was used is a
softlink at `/usr/local/bin/go`.

This change rewrites the `/usr/local/bin/go` softlink to point to the
binary at `/Users/travis/gopath/bin/go1.23.1`, so we get the right
go-version.

cmd/utils: fix `setEtherbase` (#30488)

Make `setEtherbase` fall thorugh and handle `miner.pending.feeRecipient` after showing deprecation-warning for `miner.etherbase`-flag.

core/state: fix comment of `mode` (#30490)

core/state: commit snapshot only if the base layer exists (#30493)

This pull request skips the state snapshot update if the base layer is
not existent, eliminating the numerous warning logs after an unclean
shutdown.

Specifically, Geth will rewind its chain head to a historical block
after unclean shutdown and state snapshot will be remained as unchanged
waiting for recovery. During this period of time, the snapshot is unusable
and all state updates should be ignored/skipped for state snapshot update.

internal/ethapi/api: for simulated calls, set gaspool to max value if global gascap is 0 (#30474)

In #27720, we introduced RPC global gas cap. A value of `0` means an unlimited gas cap. However, this was not the case for simulated calls. This PR fixes the behaviour.

core/rawdb: make sure specified state scheme is valid (#30499)

This change exits with error if user provided a `--state.scheme` which is neither `hash` nor `path`

internal/ethapi: fix gascap 0 for eth_simulateV1 (#30496)

Similar to #30474.

core/tracing, core/vm: add ContractCode to the OpContext (#30466)

Extends the opcontext interface to include accessor for code being executed in current context. While it is possible to get the code via `statedb.GetCode`, that approach doesn't work for initcode.

core/vm: more benchmarks for bls g1/g2-multiexp precompiles (#30459)

This change adds more comprehensive benchmarks with a wider-variety of input sizes for g1 and g2 multi exponentiation.

p2p/discover: fix flaky tests writing to test.log after completion (#30506)

This PR fixes two tests, which had a tendency to sometimes write to the `*testing.T` `log` facility after the test function had completed, which is not allowed. This PR fixes it by using waitgroups to ensure that the handler/logwriter terminates before the test exits.

closes #30505

deps: update supranational/blst (#30504)

This update should only affect the fuzzers, as far as I know. But it
seems like it might also fix some arm/macos compilation issue in
ethereum/go-ethereum#30494

Closes #30494 (I think)

core/txpool, eth/catalyst: ensure gas tip retains current value upon rollback (#30495)

Here we move the method that drops all transactions by temporarily increasing the fee
into the TxPool itself. It's better to have it there because we can set it back to the
configured value afterwards. This resolves a TODO in the simulated backend.

core/txpool/blobpool: revert part of #30437, return all reinject-addresses

core/txpool/blobpool: add test to check internal shuffling

Revert "core/txpool, eth/catalyst:  ensure gas tip retains current value upon rollback" (#30521)

Reverts ethereum/go-ethereum#30495

You are free to create a proper Clear method if that's the best way. But
one that does a proper cleanup, not some hacky call to set gas which
screws up logs, metrics and everything along the way. Also doesn't work
for legacy pool local transactions.

The current code had a hack in the simulated code, now we have a hack in
live txpooling code. No, that's not acceptable. I want the live code to
be proper, meaningful API, meaningful comments, meaningful
implementation.

params: release Geth v1.14.10

Author: gballet

.travis.yml

@@ -85,12 +85,13 @@ jobs:
       if: type = push
       os: osx
       osx_image: xcode14.2
-      go: 1.23.x
+      go: 1.23.1 # See https://github.com/ethereum/go-ethereum/pull/30478
       env:
         - azure-osx
       git:
         submodules: false # avoid cloning ethereum/tests
       script:
+        - ln -sf /Users/travis/gopath/bin/go1.23.1 /usr/local/bin/go # Work around travis go-setup bug
         - go run build/ci.go install -dlgo
         - go run build/ci.go archive -type tar -signer OSX_SIGNING_KEY -signify SIGNIFY_KEY -upload gethstore/builds
         - go run build/ci.go install -dlgo -arch arm64

beacon/engine/gen_epe.go

@@ -94,13 +94,22 @@ type executableDataMarshaling struct {
 	ExcessBlobGas *hexutil.Uint64
 }
 
+// StatelessPayloadStatusV1 is the result of a stateless payload execution.
+type StatelessPayloadStatusV1 struct {
+	Status          string      `json:"status"`
+	StateRoot       common.Hash `json:"stateRoot"`
+	ReceiptsRoot    common.Hash `json:"receiptsRoot"`
+	ValidationError *string     `json:"validationError"`
+}
+
 //go:generate go run github.com/fjl/gencodec -type ExecutionPayloadEnvelope -field-override executionPayloadEnvelopeMarshaling -out gen_epe.go
 
 type ExecutionPayloadEnvelope struct {
 	ExecutionPayload *ExecutableData `json:"executionPayload"  gencodec:"required"`
 	BlockValue       *big.Int        `json:"blockValue"  gencodec:"required"`
 	BlobsBundle      *BlobsBundleV1  `json:"blobsBundle"`
 	Override         bool            `json:"shouldOverrideBuilder"`
+	Witness          *hexutil.Bytes  `json:"witness"`
 }
 
 type BlobsBundleV1 struct {
@@ -115,9 +124,10 @@ type executionPayloadEnvelopeMarshaling struct {
 }
 
 type PayloadStatusV1 struct {
-	Status          string       `json:"status"`
-	LatestValidHash *common.Hash `json:"latestValidHash"`
-	ValidationError *string      `json:"validationError"`
+	Status          string         `json:"status"`
+	Witness         *hexutil.Bytes `json:"witness"`
+	LatestValidHash *common.Hash   `json:"latestValidHash"`
+	ValidationError *string        `json:"validationError"`
 }
 
 type TransitionConfigurationV1 struct {
@@ -198,6 +208,20 @@ func decodeTransactions(enc [][]byte) ([]*types.Transaction, error) {
 // Withdrawals value will propagate through the returned block. Empty
 // Withdrawals value must be passed via non-nil, length 0 value in data.
 func ExecutableDataToBlock(data ExecutableData, versionedHashes []common.Hash, beaconRoot *common.Hash) (*types.Block, error) {
+	block, err := ExecutableDataToBlockNoHash(data, versionedHashes, beaconRoot)
+	if err != nil {
+		return nil, err
+	}
+	if block.Hash() != data.BlockHash {
+		return nil, fmt.Errorf("blockhash mismatch, want %x, got %x", data.BlockHash, block.Hash())
+	}
+	return block, nil
+}
+
+// ExecutableDataToBlockNoHash is analogous to ExecutableDataToBlock, but is used
+// for stateless execution, so it skips checking if the executable data hashes to
+// the requested hash (stateless has to *compute* the root hash, it's not given).
+func ExecutableDataToBlockNoHash(data ExecutableData, versionedHashes []common.Hash, beaconRoot *common.Hash) (*types.Block, error) {
 	txs, err := decodeTransactions(data.Transactions)
 	if err != nil {
 		return nil, err
@@ -267,13 +291,10 @@ func ExecutableDataToBlock(data ExecutableData, versionedHashes []common.Hash, b
 		ParentBeaconRoot: beaconRoot,
 		RequestsHash:     requestsHash,
 	}
-	block := types.NewBlockWithHeader(header)
-	block = block.WithBody(types.Body{Transactions: txs, Uncles: nil, Withdrawals: data.Withdrawals, Requests: requests})
-	block = block.WithWitness(data.ExecutionWitness)
-	if block.Hash() != data.BlockHash {
-		return nil, fmt.Errorf("blockhash mismatch, want %x, got %x", data.BlockHash, block.Hash())
-	}
-	return block, nil
+	return types.NewBlockWithHeader(header).
+			WithBody(types.Body{Transactions: txs, Uncles: nil, Withdrawals: data.Withdrawals, Requests: requests}).
+			WithWitness(data.ExecutionWitness),
+		nil
 }
 
 // BlockToExecutableData constructs the ExecutableData structure by filling the

beacon/engine/types.go

@@ -156,7 +156,6 @@ var (
 		utils.BeaconGenesisRootFlag,
 		utils.BeaconGenesisTimeFlag,
 		utils.BeaconCheckpointFlag,
-		utils.CollectWitnessFlag,
 	}, utils.NetworkFlags, utils.DatabaseFlags)
 
 	rpcFlags = []cli.Flag{

cmd/geth/main.go

@@ -601,11 +601,6 @@ var (
 		Usage:    "Disables db compaction after import",
 		Category: flags.LoggingCategory,
 	}
-	CollectWitnessFlag = &cli.BoolFlag{
-		Name:     "collectwitness",
-		Usage:    "Enable state witness generation during block execution. Work in progress flag, don't use.",
-		Category: flags.MiscCategory,
-	}
 
 	// MISC settings
 	SyncTargetFlag = &cli.StringFlag{
@@ -1361,7 +1356,6 @@ func MakeAddress(ks *keystore.KeyStore, account string) (accounts.Account, error
 func setEtherbase(ctx *cli.Context, cfg *ethconfig.Config) {
 	if ctx.IsSet(MinerEtherbaseFlag.Name) {
 		log.Warn("Option --miner.etherbase is deprecated as the etherbase is set by the consensus client post-merge")
-		return
 	}
 	if !ctx.IsSet(MinerPendingFeeRecipientFlag.Name) {
 		return
@@ -1817,9 +1811,6 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) {
 		// TODO(fjl): force-enable this in --dev mode
 		cfg.EnablePreimageRecording = ctx.Bool(VMEnableDebugFlag.Name)
 	}
-	if ctx.IsSet(CollectWitnessFlag.Name) {
-		cfg.EnableWitnessCollection = ctx.Bool(CollectWitnessFlag.Name)
-	}
 
 	if ctx.IsSet(RPCGlobalGasCapFlag.Name) {
 		cfg.RPCGasCap = ctx.Uint64(RPCGlobalGasCapFlag.Name)
@@ -2252,7 +2243,6 @@ func MakeChain(ctx *cli.Context, stack *node.Node, readonly bool) (*core.BlockCh
 	}
 	vmcfg := vm.Config{
 		EnablePreimageRecording: ctx.Bool(VMEnableDebugFlag.Name),
-		EnableWitnessCollection: ctx.Bool(CollectWitnessFlag.Name),
 	}
 	if ctx.IsSet(VMTraceFlag.Name) {
 		if name := ctx.String(VMTraceFlag.Name); name != "" {

cmd/utils/flags.go

@@ -20,10 +20,8 @@ import (
 	"errors"
 	"fmt"
 
-	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/consensus"
 	"github.com/ethereum/go-ethereum/core/state"
-	"github.com/ethereum/go-ethereum/core/stateless"
 	"github.com/ethereum/go-ethereum/core/types"
 	"github.com/ethereum/go-ethereum/params"
 	"github.com/ethereum/go-ethereum/trie"
@@ -160,28 +158,6 @@ func (v *BlockValidator) ValidateState(block *types.Block, statedb *state.StateD
 	return nil
 }
 
-// ValidateWitness cross validates a block execution with stateless remote clients.
-//
-// Normally we'd distribute the block witness to remote cross validators, wait
-// for them to respond and then merge the results. For now, however, it's only
-// Geth, so do an internal stateless run.
-func (v *BlockValidator) ValidateWitness(witness *stateless.Witness, receiptRoot common.Hash, stateRoot common.Hash) error {
-	// Run the cross client stateless execution
-	// TODO(karalabe): Self-stateless for now, swap with other clients
-	crossReceiptRoot, crossStateRoot, err := ExecuteStateless(v.config, witness)
-	if err != nil {
-		return fmt.Errorf("stateless execution failed: %v", err)
-	}
-	// Stateless cross execution suceeeded, validate the withheld computed fields
-	if crossReceiptRoot != receiptRoot {
-		return fmt.Errorf("cross validator receipt root mismatch (cross: %x local: %x)", crossReceiptRoot, receiptRoot)
-	}
-	if crossStateRoot != stateRoot {
-		return fmt.Errorf("cross validator state root mismatch (cross: %x local: %x)", crossStateRoot, stateRoot)
-	}
-	return nil
-}
-
 // CalcGasLimit computes the gas limit of the next block after parent. It aims
 // to keep the baseline gas close to the provided target, and increase it towards
 // the target if the baseline gas is lower.

core/block_validator.go

@@ -201,7 +201,7 @@ func testHeaderVerificationForMerging(t *testing.T, isClique bool) {
 			t.Fatalf("post-block %d: unexpected result returned: %v", i, result)
 		case <-time.After(25 * time.Millisecond):
 		}
-		chain.InsertBlockWithoutSetHead(postBlocks[i])
+		chain.InsertBlockWithoutSetHead(postBlocks[i], false)
 	}
 
 	// Verify the blocks with pre-merge blocks and post-merge blocks