-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Closed
Description
🗺 What's left for release
Monitoring and Resource Management
- Set data-driven default resource limits go-libp2p-resource-manager#32 and rewrite limits to allow auto-scaling go-libp2p-resource-manager#48
- Defend against eclipse attacks with ALLOW-list support go-libp2p-resource-manager#29 and feat: Allowlist go-libp2p-resource-manager#47
- This includes the relevant configuration as well
- feat: Usable out of the box metrics go-libp2p-resource-manager#50 and feat: Out of the box metrics for resource manager go-libp2p-resource-manager#54
- Add canonical peer status logs #1624
Swarm Sync Notification
Misc
- update go-yamux (at least to v3.1.2), but better to a version that using go-log
- BufferPool.Get doesn't detect negative length values and blindly converts them into uint32(negative) which can trigger out of memory issues go-buffer-pool#26
- concurrent map iteration and map write in debug logging #1599
- remove legacy mDNS implementation #1192
🔦 Highlights
Resource Management
- Default resource management config limits will now autoscale depending on the machine size.
- You can now specify an allowlisted set of multiaddrs that can still connect to you even if you've reached your normal limits (still limited by the allowlist limits).
- Resource manager now exposes opencensus metrics and provides a default grafana dashboard.
Swarm Notifications are now synchronous
If your protocol depended on being given notifications in an async context and taking its time processing that, you will slow down your whole go-libp2p application. See #1562 for more details. This only matters if you implemented a network.Notifiee
and registered to receive notifications on swarm network events.
Changelog
- github.com/libp2p/go-libp2p:
- Release v0.21.0 RC (Release v0.21.0 RC #1638) (libp2p/go-libp2p#1638)
- fix: return the best acceptable conn in NewStream (fix: return the best _acceptable_ conn in NewStream #1604) (libp2p/go-libp2p#1604)
- use autoscaling limits (rcmgr: use autoscaling limits #1637) (libp2p/go-libp2p#1637)
- docs: point to SetDefaultServiceLimits in ResourceManager option (docs: point to SetDefaultServiceLimits in ResourceManager option #1636) (libp2p/go-libp2p#1636)
- chore: update deps (chore: update deps #1634) (libp2p/go-libp2p#1634)
- Pass endpoint information to resource manager's OpenConnection (Pass endpoint information to resource manager's OpenConnection #1633) (libp2p/go-libp2p#1633)
- Add canonical peer status logs (Add canonical peer status logs #1624) (libp2p/go-libp2p#1624)
- move go-libp2p-circuit here (libp2p/go-libp2p#1626)
- swarm: fix logging of accepted connections (swarm: fix logging of accepted connections #1629) (libp2p/go-libp2p#1629)
- fix: deny connections to peers in the right place (fix: deny connections to peers in the right place #1627) (libp2p/go-libp2p#1627)
- ping: fix flaky test (ping: fix flaky test #1617) (libp2p/go-libp2p#1617)
- chore: update github.com/multiformats/go-multistream to v0.3.3 (chore: update github.com/multiformats/go-multistream to v0.3.3 #1621) (libp2p/go-libp2p#1621)
- chore: use the new multiaddr.Contains function (chore: use the new multiaddr.Contains function #1618) (libp2p/go-libp2p#1618)
- chore: stop using the deprecated mux.MuxedConn (chore: stop using the deprecated mux.MuxedConn #1614) (libp2p/go-libp2p#1614)
- logging: Add canonical log for misbehaving peers (logging: Add canonical log for misbehaving peers #1600) (libp2p/go-libp2p#1600)
- use multiaddr ipcidr to parse multiaddr filters (use multiaddr ipcidr to parse multiaddr filters #1606) (libp2p/go-libp2p#1606)
- tcp: unexport TcpTransport.Upgrader (tcp: unexport TcpTransport.Upgrader #1596) (libp2p/go-libp2p#1596)
- chore: update github.com/multiformats/go-multistream to v0.3.2 (chore: update github.com/multiformats/go-multistream to v0.3.2 #1610) (libp2p/go-libp2p#1610)
- muxer: expose func to create MuxedConn from backing Conn (muxer: expose func to create MuxedConn from backing Conn #1609) (libp2p/go-libp2p#1609)
- remove legacy mDNS implementation (remove legacy mDNS implementation #1192) (libp2p/go-libp2p#1192)
- feat: allow dialing wss peers using DNS multiaddrs
- fix natManager to close natManager.nat (fix natManager to close natManager.nat #1468) (libp2p/go-libp2p#1468)
- Expose DefaultPerPeerRateLimit as var (Added perPeerRateLimit env config #1580) (libp2p/go-libp2p#1580)
- swarm: add ListenClose (swarm: add ListenClose #1586) (libp2p/go-libp2p#1586)
- identify: Fix flaky tests (identify: Fix flaky tests #1555) (libp2p/go-libp2p#1555)
- autonat: fix flaky TestAutoNATPrivate (autonat: fix flaky TestAutoNATPrivate #1581) (libp2p/go-libp2p#1581)
- pstoremanager: fix test timeout (pstoremanager: fix test timeout #1588) (libp2p/go-libp2p#1588)
- swarm: send notifications synchronously (swarm: send notifications synchronously #1562) (libp2p/go-libp2p#1562)
- basichost: fix flaky TestSignedPeerRecordWithNoListenAddrs (basichost: fix flaky TestSignedPeerRecordWithNoListenAddrs #1559) (libp2p/go-libp2p#1559)
- identify: fix flaky TestIdentifyDeltaOnProtocolChange (again) (identify: fix flaky TestIdentifyDeltaOnProtocolChange (again) #1582) (libp2p/go-libp2p#1582)
- tls: fix flaky TestInvalidCerts on Windows (libp2p/go-libp2p#1560)
- chore: log autorelay start failure error (libp2p/go-libp2p#1583)
- Add sanity check assertion (connmgr: Add sanity check to test #1570) (libp2p/go-libp2p#1570)
- swarm: speed up the TestDialWorkerLoopConcurrentFailureStress test (swarm: speed up the TestDialWorkerLoopConcurrentFailureStress test #1573) (libp2p/go-libp2p#1573)
- chore: update examples to go-libp2p v0.20.0 (chore: update examples to go-libp2p v0.20.0 #1557) (libp2p/go-libp2p#1557)
- Wait a couple seconds for ID event (identify: fix flaky test #1568) (libp2p/go-libp2p#1568)
- remove workspace and packages section from README (remove workspace and packages section from README #1563) (libp2p/go-libp2p#1563)
- fix: mkreleaselog exclude autogenerated files (fix: mkreleaselog exclude autogenerated files #1567) (libp2p/go-libp2p#1567)
- move resource manager integration tests to p2p/test/ (move resource manager integration tests to p2p/test/ #1561) (libp2p/go-libp2p#1561)
- swarm: only dial a single transport in TestDialWorkerLoopBasic (swarm: only dial a single transport in TestDialWorkerLoopBasic #1526) (libp2p/go-libp2p#1526)
- github.com/ipfs/go-cid (v0.1.0 -> v0.2.0):
- fix: remove invalid multicodec2string mappings (Dialer.Dial takes remote peerID but does not confirm that it is peer that it connected to #137) (ipfs/go-cid#137)
- update .github/workflows/go-check.yml
- update .github/workflows/go-test.yml
- update .github/workflows/automerge.yml
- bump go.mod to Go 1.17 and run go fix
- Benchmark existing ways to check for
IDENTITY
CIDs - avoid double alloc in NewCidV1
- github.com/libp2p/go-buffer-pool (v0.0.2 -> v0.1.0):
- release v0.1.0 (Allow users to override the IPFS API address when fetching dependencies #30) (libp2p/go-buffer-pool#30)
- panic if a negative length is passed to BufferPool.Get (fix #27, bad link #28) (libp2p/go-buffer-pool#28)
- test: fix gc test on go 1.16 (libp2p/go-buffer-pool#18)
- fix staticcheck (libp2p/go-buffer-pool#16)
- ci: bump go to 1.13 (libp2p/go-buffer-pool#13)
- test: make sure we have the correct number of pools (libp2p/go-buffer-pool#10)
- github.com/libp2p/go-libp2p-core (v0.16.1 -> v0.19.0):
- Release v0.19.0 (TestMdnsDiscovery fails #271) (libp2p/go-libp2p-core#271)
- Add endpoint parameter to the OpenConnection method for ResourceManager (Switch mdns library #257) (libp2p/go-libp2p-core#257)
- Release v0.18.0 (Please Git tag releases #270) (libp2p/go-libp2p-core#270)
- Add canonical peer status logging with sampling (make the identify test less racy #269) (libp2p/go-libp2p-core#269)
- canonicallog: reduce log level to warning (add link to example implementation #268) (libp2p/go-libp2p-core#268)
- Only log once if we failed to convert from netAddr (Added default muxer in case when none was provided. #264) (libp2p/go-libp2p-core#264)
- remove deprecated mux package (fix peerstore apocalypse redux #265) (libp2p/go-libp2p-core#265)
- chore: upgrade to go-cid to v0.2.0 (dial attempt failed: bind: address already in use #262) (libp2p/go-libp2p-core#262)
- remove the peer.Set (gx: update go-libp2p-peerstore #261) (libp2p/go-libp2p-core#261)
- Bump version (Please provide more examples #259) (libp2p/go-libp2p-core#259)
- Add canonical log for misbehaving peers (Echo example: yamux potentially broken #258) (libp2p/go-libp2p-core#258)
- github.com/libp2p/go-libp2p-peerstore (v0.6.0 -> v0.7.1):
- Release v0.7.1 (libp2p/go-libp2p-peerstore#202)
- stop using the peer.Set (gx: update go-addr-util, go-libp2p-swarm #201) (libp2p/go-libp2p-peerstore#201)
- feat: Use a clock interface in pstoreds as well (libp2p/go-libp2p-peerstore#200)
- feat: use a clock interface to better support testing for pstoremem (libp2p/go-libp2p-peerstore#199)
- pstoremem: fix slice preallocation in GetProtocols (How to install deps? #198) (libp2p/go-libp2p-peerstore#198)
- remove all calls to peer.ID.Validate (libp2p/go-libp2p-peerstore#194)
- remove the addr package (libp2p/go-libp2p-peerstore#195)
- move AddrList to pstoremen, unexport it (libp2p/go-libp2p-peerstore#193)
- optimize allocations in the memory address book (libp2p/go-libp2p-peerstore#191)
- implement a clean shutdown for the memory address book (libp2p/go-libp2p-peerstore#192)
- github.com/libp2p/go-libp2p-resource-manager (v0.3.0 -> v0.5.0):
- release v0.5.0 (libp2p/go-libp2p-resource-manager#60)
- Add docs around WithAllowlistedMultiaddrs. Expose allowlist (libp2p/go-libp2p-resource-manager#63)
- fix marshalling of allowlisted scopes (libp2p/go-libp2p-resource-manager#62)
- docs: describe how the limiter is configured, and how limits are scaled (update to official yamux fixes #59) (libp2p/go-libp2p-resource-manager#59)
- don't limit the number of FDs on Windows (don't leak goroutines when pings complete #58) (libp2p/go-libp2p-resource-manager#58)
- Add ability to configure allowlist limits (libp2p/go-libp2p-resource-manager#57)
- rewrite limits to allow auto-scaling (libp2p/go-libp2p-resource-manager#48)
- Release v0.4.0 (libp2p/go-libp2p-resource-manager#56)
- feat: Out of the box metrics for resource manager (libp2p/go-libp2p-resource-manager#54)
- feat: Allowlist (libp2p/go-libp2p-resource-manager#47)
- trace the scope as a JSON object (gx publish version 3.2.1 #52) (libp2p/go-libp2p-resource-manager#52)
- include current limits in debug messages (libp2p/go-libp2p-resource-manager#42)
- add an ID to spans (add a read deadline for the multistream select on dial #44) (libp2p/go-libp2p-resource-manager#44)
- add a DefaultLimitConfig with infinite limits (libp2p spec link in CONTRIBUTE is a broken link. #41) (libp2p/go-libp2p-resource-manager#41)
- export the TraceEvt (Publish 3.0.0 #40) (libp2p/go-libp2p-resource-manager#40)
- trace exact timestamps (Bump to 2.0.3 #39) (libp2p/go-libp2p-resource-manager#39)
- skip events that don't change anything in tracer (refactor swarm dialing logic #38) (libp2p/go-libp2p-resource-manager#38)
- fix typos in MetricsReporter docs
- fix shadowing of service name (Deps/cleanup #37) (libp2p/go-libp2p-resource-manager#37)
- add a timestamp to trace events (abstraction for transport level streams #34) (libp2p/go-libp2p-resource-manager#34)
- github.com/libp2p/go-libp2p-testing (v0.9.2 -> v0.10.0):
- Release v0.10.0 (libp2p/go-libp2p-testing#62)
- Update go-libp2p-core dep (libp2p/go-libp2p-testing#61)
- remove suites (split peerstore from peer package #60) (libp2p/go-libp2p-testing#60)
- don't continue on read / write error in stream suite (update to official yamux fixes #59) (libp2p/go-libp2p-testing#59)
- remove debug logging from stream and muxer suite (libp2p/go-libp2p-testing#58)
- remove Travis package (Ping command creates a lot of goroutines #57) (libp2p/go-libp2p-testing#57)
- github.com/lucas-clemente/quic-go (v0.27.1 -> v0.28.0):
- update for Go 1.19beta1 (#3460) (lucas-clemente/quic-go#3460)
- dedupe Alt-Svc header values (#3461) (lucas-clemente/quic-go#3461)
- only set DF for sockets that can handle it (#3448) (lucas-clemente/quic-go#3448)
- fix flaky HTTP/3 request body test (#3447) (lucas-clemente/quic-go#3447)
- make the keep alive interval configurable (#3444) (lucas-clemente/quic-go#3444)
- implement QUIC v2 (lucas-clemente/quic-go#3432)
- allow HTTP clients and servers to take over the request stream (lucas-clemente/quic-go#3437)
- remove the http3.DataStreamer (#3435) (lucas-clemente/quic-go#3435)
- update qtls to include the recent session ticket changes to crypto/tls (#3443) (lucas-clemente/quic-go#3443)
- always reset header buffer, even when QPACK encoding fails (#3436) (lucas-clemente/quic-go#3436)
- Change "HTTP/3" to "HTTP/3.0". (#3439) (lucas-clemente/quic-go#3439)
- remove stray http3 connection file
- pass frame / stream type parsing errors to the hijacker callbacks (lucas-clemente/quic-go#3429)
- add test for bidirectional stream hijacker (#3434) (lucas-clemente/quic-go#3434)
- make it possible to parse a varint at the end of a reader (#3428) (lucas-clemente/quic-go#3428)
- don't ignore errors that occur when the TLS ClientHello is generated (lucas-clemente/quic-go#3424)
- don't send path MTU probe packets on a timer (#3423) (lucas-clemente/quic-go#3423)
- introduce a http3.RoundTripOpt to prevent closing of request stream (#3411) (lucas-clemente/quic-go#3411)
- don't close the request stream when http3.DataStreamer was used (#3413) (lucas-clemente/quic-go#3413)
- do not embed http.Server in http3.Server (#3397) (lucas-clemente/quic-go#3397)
- remove error return value from ComposeVersionNegotiation (#3410) (lucas-clemente/quic-go#3410)
- don't set receive buffer if it is already large enough (#3407) (lucas-clemente/quic-go#3407)
- clone TLS conf in newClient (#3400) (lucas-clemente/quic-go#3400)
- remove warning comments of stable implementation (#3399) (lucas-clemente/quic-go#3399)
- fix parsing of request path for Extended CONNECT requests (#3388) (lucas-clemente/quic-go#3388)
- update docs to reflect that we support RFC 9221 (Unreliable Datagrams) (#3382) (lucas-clemente/quic-go#3382)
- fix deadlock on concurrent http3.Server.Serve and Close calls (#3387) (lucas-clemente/quic-go#3387)
- reduce flakiness of deadline integration tests (#3383) (lucas-clemente/quic-go#3383)
- protect against concurrent use of Stream.Write (#3381) (lucas-clemente/quic-go#3381)
- protect against concurrent use of Stream.Read (#3380) (lucas-clemente/quic-go#3380)
- Expose quic server closed err (#3395) (lucas-clemente/quic-go#3395)
- implement HTTP/3 unidirectional stream hijacking (#3389) (lucas-clemente/quic-go#3389)
- add LocalAddr and RemoteAddr functions to http3.StreamCreator (#3384) (lucas-clemente/quic-go#3384)
- extend the HTTP/3 API for WebTransport support (lucas-clemente/quic-go#3362)
- remove unneeded network from custom dial function used in HTTP/3 (Fix:QUIC connection rate limiting with DoS protection #3368) (lucas-clemente/quic-go#3368)
- github.com/multiformats/go-multicodec (v0.4.1 -> v0.5.0):
- Bump version to 0.5.0
- Bump version to 0.4.2
- deps: update stringer version in go generate command
- update .github/workflows/go-check.yml
- update .github/workflows/go-test.yml
- update .github/workflows/automerge.yml
- bump go.mod to Go 1.17 and run go fix
- docs(readme): improved usage examples (p2p/net/swarm: Fails to build due to dependency vendoring dependencies #66) (multiformats/go-multicodec#66)
Contributors
Contributor | Commits | Lines ± | Files Changed |
---|---|---|---|
Marten Seemann | 217 | +85564/-25980 | 1143 |
Marco Munizaga | 78 | +6722/-2489 | 150 |
vyzo | 64 | +6393/-666 | 130 |
Steven Allen | 31 | +1357/-745 | 54 |
Jeromy | 5 | +1032/-64 | 16 |
Jorropo | 6 | +168/-33 | 14 |
hareku | 3 | +172/-6 | 7 |
Marcin Rataj | 2 | +50/-124 | 5 |
Ivan Trubach | 1 | +98/-74 | 6 |
Jakub Sztandera | 1 | +101/-62 | 1 |
Christian Stewart | 3 | +89/-48 | 14 |
Raúl Kripalani | 2 | +69/-62 | 9 |
Masih H. Derkani | 3 | +84/-13 | 5 |
Julien Muret | 1 | +60/-7 | 2 |
Lars Gierth | 1 | +20/-29 | 4 |
Cole Brown | 4 | +27/-19 | 4 |
Chao Fei | 2 | +15/-30 | 9 |
Nuno Diegues | 2 | +25/-18 | 9 |
Daniel Martí | 1 | +15/-6 | 1 |
Wiktor Jurkiewicz | 1 | +13/-5 | 1 |
Matt Robenolt | 1 | +15/-1 | 2 |
gammazero | 1 | +7/-6 | 3 |
aarshkshah1992 | 2 | +8/-2 | 2 |
Aaron Riekenberg | 1 | +4/-4 | 4 |
Rod Vagg | 3 | +3/-3 | 3 |
Adrian Lanzafame | 1 | +3/-3 | 1 |
Dmitriy Ryajov | 2 | +2/-3 | 2 |
millken | 1 | +1/-1 | 1 |
Matt Joiner | 1 | +1/-1 | 1 |
Leo Balduf | 1 | +1/-1 | 1 |
Didrik Nordström | 1 | +2/-0 | 1 |
Adin Schmahmann | 1 | +1/-1 | 1 |
✅ Release Checklist
- Stage 0 - Finishing Touches
- Go through relevant libp2p repos looking for unreleased changes that should make it into the release. If you find any, cut releases.
- Run
go get -u ./...
to see if there are any out-of-date deps that look important. If there are, bubble them. Try to avoid directly updating indirect deps in go-libp2p's go.mod when possible. - Make sure local tests are passing.
- Stage 1 - Upstream Testing
- Create testing branches in lotus & go-ipfs with the new go-libp2p release and run CI/tests. Many upstream projects are tested in CI, but lotus & go-ipfs are not.
- (someday) Run upstream testground tests. Unfortunately, this is too time consuming at the moment.
- (someday) Run bitswap testground tests.
- (someday) Run DHT testground tests.
- Stage 2 - Infrastructure Testing
- How: Using the testing branches created above, work with the infrastructure team to deploy the new libp2p versions to our infrastructure.
- Where:
- A go-ipfs gateway.
- Deploy
- Analyze
- Look at pprof profile dumps, especially CPU profiles and heap allocation profiles, noting any significant changes.
- A go-ipfs bootstrapper.
- Deploy
- Analyze
- Look at pprof profile dumps, especially CPU profiles and heap allocation profiles, noting any significant changes.
- Check peers (e.g.,
ipfs swarm peers
) to make sure we're connecting to peers on all transports. - Check advertised addresses and protocols (e.g.,
ipfs id
) to make sure they're sane.
- A go-ipfs gateway.
- Stage 3 - Release
- Tag the release on master.
- Publish the release through the GitHub UI, adding the release notes. Some users rely on this to receive notifications of new releases.
- Announce the release on the discuss.libp2p.io.
- Stage 4 - Update Upstream
- Update the examples to the final release
- Update the upstream testing branches to the final release and create PRs.
- Make required changes to the release process.
Metadata
Metadata
Assignees
Labels
No labels