Skip to content

Commit 58d6440

Browse files
steezeburgerjoroshiba
authored andcommitted
Feature/grpc execution api (#1)
* add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (ethereum#26976) core/state, trie: remove Try prefix in Trie accessors (ethereum#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <[email protected]> metrics/librato: ensure resp.body closed (ethereum#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (ethereum#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <[email protected]> Co-authored-by: Martin Holst Swende <[email protected]> core/bloombits: use atomic type (ethereum#26993) core/state: use atomic.Bool (ethereum#26992) graphql: fix data races (ethereum#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (ethereum#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <[email protected]> internal/debug: add log.logfmt flag to set logging to use logfmt (ethereum#26970) docs: update outdated DeriveSha docs comment (ethereum#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <[email protected]> Co-authored-by: elizabeth <[email protected]> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <[email protected]> logging remove unused dependency Mark the chain 'finalized' on startup
1 parent 916d6a4 commit 58d6440

File tree

30 files changed

+2885
-70
lines changed

30 files changed

+2885
-70
lines changed
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: Build and Publish Docker image
2+
3+
# Trigger on pushes to astria branch, new semantic version tags, and pull request updates
4+
on:
5+
workflow_dispatch:
6+
push:
7+
branches:
8+
- "astria"
9+
tags:
10+
- "v[0-9]+.[0-9]+.[0-9]+"
11+
- "v[0-9]+.[0-9]+.[0-9]+-alpha.[0-9]+"
12+
- "v[0-9]+.[0-9]+.[0-9]+-beta.[0-9]+"
13+
- "v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
14+
# trigger on pull request updates when target is `astria` branch
15+
pull_request:
16+
branches:
17+
- "astria"
18+
19+
jobs:
20+
build-and-publish-latest:
21+
runs-on: ubuntu-latest
22+
steps:
23+
# Checking out the repo
24+
- uses: actions/checkout@v3
25+
# Setting up Go
26+
- uses: actions/setup-go@v4
27+
with:
28+
go-version: "^1.20.x" # The Go version to download (if necessary) and use.
29+
- run: go version
30+
31+
# https://github.com/docker/setup-qemu-action
32+
- name: Set up QEMU
33+
uses: docker/setup-qemu-action@v2
34+
# https://github.com/docker/setup-buildx-action
35+
- name: Set up Docker Buildx
36+
id: buildx
37+
uses: docker/setup-buildx-action@v2
38+
- name: Log in to GitHub Container Registry
39+
uses: docker/login-action@v2
40+
with:
41+
registry: ghcr.io
42+
username: ${{ github.actor }}
43+
password: ${{ secrets.GITHUB_TOKEN }}
44+
# Generate correct tabs and labels
45+
- name: Docker metadata
46+
id: metadata
47+
uses: docker/metadata-action@v4
48+
with:
49+
images: |
50+
ghcr.io/astriaorg/go-ethereum
51+
tags: |
52+
type=ref,event=pr
53+
type=semver,pattern={{major}}.{{minor}}.{{patch}}
54+
type=sha
55+
# set latest tag for `astria` branch
56+
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'astria') }}
57+
- name: Build and push
58+
uses: docker/build-push-action@v4
59+
with:
60+
# this gets rid of the unknown/unknown image that is created without this setting
61+
# https://github.com/docker/build-push-action/issues/820#issuecomment-1455687416
62+
provenance: false
63+
context: .
64+
# It takes a long time to build the arm image right now, so we only build it on tags which is what we use for releases, or on merges to the default branch.
65+
platforms: ${{ (contains(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/astria') && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
66+
push: true
67+
tags: ${{ steps.metadata.outputs.tags }}
68+
labels: ${{ steps.metadata.outputs.labels }}

Dockerfile

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,14 @@ RUN cd /go-ethereum && go run build/ci.go install -static ./cmd/geth
2020
FROM alpine:latest
2121

2222
RUN apk add --no-cache ca-certificates
23+
# Astria - add bash and jq to support start-geth.sh in conductor
24+
RUN apk add bash jq
25+
# Astria - copy genesis.json so it can be used in start-geth.sh
26+
COPY genesis.json /genesis.json
2327
COPY --from=builder /go-ethereum/build/bin/geth /usr/local/bin/
2428

25-
EXPOSE 8545 8546 30303 30303/udp
29+
# Astria - add 50051 for GRPC
30+
EXPOSE 8545 8546 30303 30303/udp 50051
2631
ENTRYPOINT ["geth"]
2732

2833
# Add some metadata labels to help programatic image consumption

Dockerfile.alltools

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ FROM alpine:latest
2222
RUN apk add --no-cache ca-certificates
2323
COPY --from=builder /go-ethereum/build/bin/* /usr/local/bin/
2424

25-
EXPOSE 8545 8546 30303 30303/udp
25+
# Astria - add 50051 for GRPC
26+
EXPOSE 8545 8546 30303 30303/udp 50051
2627

2728
# Add some metadata labels to help programatic image consumption
2829
ARG COMMIT=""

cmd/geth/config.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ import (
3737
"github.com/ethereum/go-ethereum/eth/catalyst"
3838
"github.com/ethereum/go-ethereum/eth/downloader"
3939
"github.com/ethereum/go-ethereum/eth/ethconfig"
40+
"github.com/ethereum/go-ethereum/grpc/execution"
4041
"github.com/ethereum/go-ethereum/internal/ethapi"
4142
"github.com/ethereum/go-ethereum/internal/flags"
4243
"github.com/ethereum/go-ethereum/internal/version"
@@ -201,6 +202,14 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
201202
if ctx.IsSet(utils.GraphQLEnabledFlag.Name) {
202203
utils.RegisterGraphQLService(stack, backend, filterSystem, &cfg.Node)
203204
}
205+
206+
// Configure gRPC if requested.
207+
if ctx.IsSet(utils.GRPCEnabledFlag.Name) {
208+
serviceV1a1 := execution.NewExecutionServiceServerV1Alpha1(eth)
209+
serviceV1a2 := execution.NewExecutionServiceServerV1Alpha2(eth)
210+
utils.RegisterGRPCExecutionService(stack, serviceV1a1, serviceV1a2, &cfg.Node)
211+
}
212+
204213
// Add the Ethereum Stats daemon if requested.
205214
if cfg.Ethstats.URL != "" {
206215
utils.RegisterEthStatsService(stack, backend, cfg.Ethstats.URL)

cmd/geth/main.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,9 @@ var (
176176
utils.AllowUnprotectedTxs,
177177
utils.BatchRequestLimit,
178178
utils.BatchResponseMaxSize,
179+
utils.GRPCEnabledFlag,
180+
utils.GRPCHostFlag,
181+
utils.GRPCPortFlag,
179182
}
180183

181184
metricsFlags = []cli.Flag{

cmd/utils/flags.go

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,8 @@ import (
5555
"github.com/ethereum/go-ethereum/ethdb/remotedb"
5656
"github.com/ethereum/go-ethereum/ethstats"
5757
"github.com/ethereum/go-ethereum/graphql"
58+
executionv1a1 "github.com/ethereum/go-ethereum/grpc/gen/astria/execution/v1alpha1"
59+
executionv1a2 "github.com/ethereum/go-ethereum/grpc/gen/astria/execution/v1alpha2"
5860
"github.com/ethereum/go-ethereum/internal/ethapi"
5961
"github.com/ethereum/go-ethereum/internal/flags"
6062
"github.com/ethereum/go-ethereum/les"
@@ -733,6 +735,24 @@ var (
733735
Usage: "Enables the (deprecated) personal namespace",
734736
Category: flags.APICategory,
735737
}
738+
// grpc
739+
GRPCEnabledFlag = &cli.BoolFlag{
740+
Name: "grpc",
741+
Usage: "Enable the gRPC server",
742+
Category: flags.APICategory,
743+
}
744+
GRPCHostFlag = &cli.StringFlag{
745+
Name: "grpc.addr",
746+
Usage: "gRPC server listening interface",
747+
Value: node.DefaultGRPCHost,
748+
Category: flags.APICategory,
749+
}
750+
GRPCPortFlag = &cli.IntFlag{
751+
Name: "grpc.port",
752+
Usage: "gRPC server listening port",
753+
Value: node.DefaultGRPCPort,
754+
Category: flags.APICategory,
755+
}
736756

737757
// Network Settings
738758
MaxPeersFlag = &cli.IntFlag{
@@ -1175,6 +1195,19 @@ func setHTTP(ctx *cli.Context, cfg *node.Config) {
11751195
}
11761196
}
11771197

1198+
// setGRPC creates the gRPC RPC listener interface string from the set command
1199+
// line flags, returning empty if the gRPC endpoint is disabled.
1200+
func setGRPC(ctx *cli.Context, cfg *node.Config) {
1201+
if ctx.Bool(GRPCEnabledFlag.Name) {
1202+
if ctx.IsSet(GRPCHostFlag.Name) {
1203+
cfg.GRPCHost = ctx.String(GRPCHostFlag.Name)
1204+
}
1205+
if ctx.IsSet(GRPCPortFlag.Name) {
1206+
cfg.GRPCPort = ctx.Int(GRPCPortFlag.Name)
1207+
}
1208+
}
1209+
}
1210+
11781211
// setGraphQL creates the GraphQL listener interface string from the set
11791212
// command line flags, returning empty if the GraphQL endpoint is disabled.
11801213
func setGraphQL(ctx *cli.Context, cfg *node.Config) {
@@ -1415,6 +1448,7 @@ func SetNodeConfig(ctx *cli.Context, cfg *node.Config) {
14151448
SetP2PConfig(ctx, &cfg.P2P)
14161449
setIPC(ctx, cfg)
14171450
setHTTP(ctx, cfg)
1451+
setGRPC(ctx, cfg)
14181452
setGraphQL(ctx, cfg)
14191453
setWS(ctx, cfg)
14201454
setNodeUserIdent(ctx, cfg)
@@ -1948,6 +1982,14 @@ func RegisterGraphQLService(stack *node.Node, backend ethapi.Backend, filterSyst
19481982
}
19491983
}
19501984

1985+
// RegisterGRPCExecutionService adds the gRPC API to the node.
1986+
// It was done this way so that our grpc execution server can access the ethapi.Backend
1987+
func RegisterGRPCExecutionService(stack *node.Node, execServerV1a1 executionv1a1.ExecutionServiceServer, execServerV1a2 executionv1a2.ExecutionServiceServer, cfg *node.Config) {
1988+
if err := node.NewGRPCServerHandler(stack, execServerV1a1, execServerV1a2, cfg); err != nil {
1989+
Fatalf("Failed to register the gRPC service: %v", err)
1990+
}
1991+
}
1992+
19511993
// RegisterFilterAPI adds the eth log filtering RPC API to the node.
19521994
func RegisterFilterAPI(stack *node.Node, backend ethapi.Backend, ethcfg *ethconfig.Config) *filters.FilterSystem {
19531995
isLightClient := ethcfg.SyncMode == downloader.LightSync

consensus/ethash/consensus.go

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -234,7 +234,6 @@ func (ethash *Ethash) verifyHeader(chain consensus.ChainHeaderReader, header, pa
234234
}
235235
// Verify the block's difficulty based on its timestamp and parent's difficulty
236236
expected := ethash.CalcDifficulty(chain, header.Time, parent)
237-
238237
if expected.Cmp(header.Difficulty) != 0 {
239238
return fmt.Errorf("invalid difficulty: have %v, want %v", header.Difficulty, expected)
240239
}

core/blockchain.go

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -319,10 +319,10 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
319319
return nil, ErrNoGenesis
320320
}
321321

322-
bc.currentBlock.Store(nil)
323-
bc.currentSnapBlock.Store(nil)
324-
bc.currentFinalBlock.Store(nil)
325-
bc.currentSafeBlock.Store(nil)
322+
bc.currentBlock.Store(bc.genesisBlock.Header())
323+
bc.currentSnapBlock.Store(bc.genesisBlock.Header())
324+
bc.currentFinalBlock.Store(bc.genesisBlock.Header())
325+
bc.currentSafeBlock.Store(bc.genesisBlock.Header())
326326

327327
// Update chain info data metrics
328328
chainInfoGauge.Update(metrics.GaugeInfoValue{"chain_id": bc.chainConfig.ChainID.String()})

core/txpool/txpool.go

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,8 @@ type TxPool struct {
7272

7373
subs event.SubscriptionScope // Subscription scope to unsubscribe all on shutdown
7474
quit chan chan error // Quit channel to tear down the head updater
75+
76+
astria *astriaOrdered
7577
}
7678

7779
// New creates a new transaction pool to gather, sort and filter inbound
@@ -356,6 +358,108 @@ func (p *TxPool) Stats() (int, int) {
356358
return runnable, blocked
357359
}
358360

361+
func (pool *TxPool) SetAstriaOrdered(rawTxs [][]byte) {
362+
txs := []*types.Transaction{}
363+
for idx, rawTx := range rawTxs {
364+
tx := new(types.Transaction)
365+
err := tx.UnmarshalBinary(rawTx)
366+
if err != nil {
367+
log.Warn("failed to unmarshal raw astria tx bytes", rawTx, "at index", idx, "error:", err)
368+
continue
369+
}
370+
371+
err = pool.astriaValidate(tx)
372+
if err != nil {
373+
log.Warn("astria tx failed validation at index", idx, "error:", err)
374+
continue
375+
}
376+
377+
txs = append(txs, tx)
378+
}
379+
380+
pool.astria = newAstriaOrdered(types.Transactions(txs))
381+
}
382+
383+
func (pool *TxPool) ClearAstriaOrdered() {
384+
if pool.astria == nil {
385+
return
386+
}
387+
pool.astria.clear()
388+
}
389+
390+
func (pool *TxPool) AstriaOrdered() *types.Transactions {
391+
// sus but whatever
392+
if pool.astria == nil {
393+
return &types.Transactions{}
394+
}
395+
return &pool.astria.txs
396+
}
397+
398+
// validateTx checks whether a transaction is valid according to the consensus
399+
// rules and adheres to some heuristic limits of the local node (price and size).
400+
func (pool *TxPool) astriaValidate(tx *types.Transaction) error {
401+
// Accept only legacy transactions until EIP-2718/2930 activates.
402+
if !pool.eip2718 && tx.Type() != types.LegacyTxType {
403+
return core.ErrTxTypeNotSupported
404+
}
405+
// Reject dynamic fee transactions until EIP-1559 activates.
406+
if !pool.eip1559 && tx.Type() == types.DynamicFeeTxType {
407+
return core.ErrTxTypeNotSupported
408+
}
409+
// Reject transactions over defined size to prevent DOS attacks
410+
if tx.Size() > txMaxSize {
411+
return ErrOversizedData
412+
}
413+
// Check whether the init code size has been exceeded.
414+
if pool.shanghai && tx.To() == nil && len(tx.Data()) > params.MaxInitCodeSize {
415+
return fmt.Errorf("%w: code size %v limit %v", core.ErrMaxInitCodeSizeExceeded, len(tx.Data()), params.MaxInitCodeSize)
416+
}
417+
// Transactions can't be negative. This may never happen using RLP decoded
418+
// transactions but may occur if you create a transaction using the RPC.
419+
if tx.Value().Sign() < 0 {
420+
return ErrNegativeValue
421+
}
422+
// Ensure the transaction doesn't exceed the current block limit gas.
423+
if pool.currentMaxGas < tx.Gas() {
424+
return ErrGasLimit
425+
}
426+
// Sanity check for extremely large numbers
427+
if tx.GasFeeCap().BitLen() > 256 {
428+
return core.ErrFeeCapVeryHigh
429+
}
430+
if tx.GasTipCap().BitLen() > 256 {
431+
return core.ErrTipVeryHigh
432+
}
433+
// Ensure gasFeeCap is greater than or equal to gasTipCap.
434+
if tx.GasFeeCapIntCmp(tx.GasTipCap()) < 0 {
435+
return core.ErrTipAboveFeeCap
436+
}
437+
// Make sure the transaction is signed properly.
438+
from, err := types.Sender(pool.signer, tx)
439+
if err != nil {
440+
return ErrInvalidSender
441+
}
442+
// Ensure the transaction adheres to nonce ordering
443+
if pool.currentState.GetNonce(from) > tx.Nonce() {
444+
return core.ErrNonceTooLow
445+
}
446+
// Transactor should have enough funds to cover the costs
447+
// cost == V + GP * GL
448+
balance := pool.currentState.GetBalance(from)
449+
if balance.Cmp(tx.Cost()) < 0 {
450+
return core.ErrInsufficientFunds
451+
}
452+
// Ensure the transaction has more gas than the basic tx fee.
453+
intrGas, err := core.IntrinsicGas(tx.Data(), tx.AccessList(), tx.To() == nil, true, pool.istanbul, pool.shanghai)
454+
if err != nil {
455+
return err
456+
}
457+
if tx.Gas() < intrGas {
458+
return core.ErrIntrinsicGas
459+
}
460+
return nil
461+
}
462+
359463
// Content retrieves the data content of the transaction pool, returning all the
360464
// pending as well as queued transactions, grouped by account and sorted by nonce.
361465
func (p *TxPool) Content() (map[common.Address][]*types.Transaction, map[common.Address][]*types.Transaction) {
@@ -415,3 +519,17 @@ func (p *TxPool) Status(hash common.Hash) TxStatus {
415519
}
416520
return TxStatusUnknown
417521
}
522+
523+
type astriaOrdered struct {
524+
txs types.Transactions
525+
}
526+
527+
func newAstriaOrdered(txs types.Transactions) *astriaOrdered {
528+
return &astriaOrdered{
529+
txs: txs,
530+
}
531+
}
532+
533+
func (ao *astriaOrdered) clear() {
534+
ao.txs = *&types.Transactions{}
535+
}

eth/catalyst/api.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,7 @@ func newConsensusAPIWithoutHeartbeat(eth *eth.Ethereum) *ConsensusAPI {
172172
// If there are payloadAttributes: we try to assemble a block with the payloadAttributes
173173
// and return its payloadID.
174174
func (api *ConsensusAPI) ForkchoiceUpdatedV1(update engine.ForkchoiceStateV1, payloadAttributes *engine.PayloadAttributes) (engine.ForkChoiceResponse, error) {
175+
log.Info("ForkchoiceUpdatedV1 called")
175176
if payloadAttributes != nil {
176177
if payloadAttributes.Withdrawals != nil {
177178
return engine.STATUS_INVALID, engine.InvalidParams.With(errors.New("withdrawals not supported in V1"))

0 commit comments

Comments
 (0)