Road to 0.5

[Stebalien]

[Living]

We're working towards a pretty large 0.5.0 go-ipfs release but still have some work remaining.

ETA: April 7th because apparently April 1st +/- a few days is reserved for jokes.

Release blockers

• Working (small-data) content routing. That is, the DHT should be useful again for finding small data sets.

Landed features

• Streaming pin ls results.
• "internal" daemon plugin for perma-unstable plugins with direct access to go-ipfs internals.
• Updated QUIC transport (backwards incompatible)
  • Updates the underlying QUIC protocol to draft 27. This should be the last breaking change to QUIC before it stabilizes.
  • Updates the handshake protocol to support ed25519 keys.
• Significantly improved bitswap performance when serving data from high-latency disks.
• Systemd support
  • Socket activation
  • Example unit files
  • Systemd startup/shutdown notifications
• Support for .eth links in /ipns/.
• A new and orders of magnitude faster (than rabin) content based chunker for ipfs add (buzhash).
• Unix domain socket support for the HTTP API.
• Identify Push - Libp2p now pushes announcements of address/protocol support changes made at runtime.
• Significant extensions to the bitswap protocol PoC of Bitswap protocol extensions implementation go-bitswap#189
• Async datastore writes in badger (2x performance improvements). NEEDS CONFIGURATION.
• Write coalescing: We previously added support for batching small writes but ran into a significant bug (Write coalescing breaks against buggy secio  libp2p/go-libp2p#644) and had to pull out at the last minute. We have re-enabled this feature.
• We will now prefer the TLS security transport over SECIO.

Next Up

• QUIC by default. We've been working towards switching over to QUIC (a UDP-based transport) by default for quite some time. However, we've recently made some significant breaking changes to both the underlying protocol (bringing our implementation in-line with the draft spec) and the way in which we structure our QUIC TLS certificates.
• Badger based datastore by default. We've been testing out badger for well over a year now and plan on switching over in the near future. However:
  • Badger just released a 2.0 with some significant changes and an incompatible database format.
  • Badger 1.0, in our current configuration, uses a lot of memory.
  • We still haven't thoroughly stress-tested badger.

Todo

• Finish the remaining TODOs in testground: Unblock the testing for go-ipfs 0.5.0 release testground/testground#196
• Fix p2p circuit addresses p2p-circuit addresses are still printed with /ipfs/QmRelay/p2p-circuit/p2p/QmTarget #6868
  • Can't reproduce. May have failed to upgrade something properly.
• Bitswap refactor (see linked issue for the checklist): Path to landing Bitswap PoC #6782
• Switch to asynchronous datastore writes by default in badger. - Improve Add Performance W/ Async Datastores #6775
• Switch to /p2p multiaddrs: https://github.com/multiformats/go-multiaddr/pull/115/files
• Support base32 PIDs for IPNS.
• Base32 encode keys in the keystore
  • Feature: Introducing EncodedFSKeystore with base32 encoding (#5947) #6955
• Fix logging to files on windows: Windows Support go-log#73
• Handle bootstrapper fallout: Fixing Bootstrapping #6797
  • Fix outstanding issues.
  • Implement migration.
  • Cut a migrations release and update go-ipfs to use it.
• Add support for a libp2p event plugin (Connection/event plugin type #6827). Our current network stats (i.e., number of nodes, we don't collect IP addresses) rely on crawling the DHT. However, nodes behind NATs will no longer join the DHT. We'll have to passively count peer IDs on the bootstrappers using a custom plugin.
• Downgrade to go 1.13 to avoid scheduling/timing performance issues on go 1.14.
  • fix: downgrade to go 1.13 #7054
• TLS by default: feat: tls by default #7055
• Fix a bitswap wantlist leak Wantlist Leak go-bitswap#310
• Fix libp2p address filtering Address filtering is not working properly #6995
• Upgrade the QUIC transport (downgraded as it was incompatible with go 1.13).
• Fix the DHT: https://github.com/libp2p/go-libp2p-kad-dht/milestone/3
• Maybe, maybe badger by default. It's back on the menu because we got it down to ~20MiB of memory.

[godcong]

https://github.com/dgraph-io/badger 2.0.0 was released a period of time.
I tried to compiled it into 0.5.0.
it works well.
perhaps we can integrate it when 0.5.0 released.

I have seen this plan.

[Stebalien]

Status update from testground on unblocking the release.

[momack2]

Very glad this list exists - thank you! To help ease the tracking of the remaining burn-down items as we get closer to launch - can you add owners to each of the remaining todos and create or link tracking issues (with scope of required tests) for each of the "test with testground" items?