ExploreTrendingAnalytics
Nostr Archives
ExploreTrendingAnalytics
[DEPRECATED] nextwave32d ago
But I'm not talking specifically about the benchmarks for the encoder. There are other considerations like bandwidth (which you did mention via compression), storage, and signature verifications. Also, formatting binary into plaintext is pretty trivial for supporting development and diagnostics. I'm just trying to get at there being a reason why a bitcoin transaction, for example, is submitted and read raw. If that goes against your principles, I'm not really sure where to go from here. Other than that, nothing you said was controversial. I'm not advocating for tearing it all down, it is what it is, I'm just pointing out that there are significant legacy costs that I personally have no appetite for, and is not sticky for me, which could have been avoided with careful thought at the initial stages. But seriously though, best of luck with it. I'm convinced you will make significant improvements.
💬 1 replies

Thread context

Root: a61082a5f4c2…

Replying to: 47d9ff0c5174…

Replies (1)

mleku32d ago
Bitcoin is compact because the data is immutable and has to be stored by all nodes. Nostr only has to store data where people need it — size is not as important when eliminating redundancy is cheap and pipelined, adding mere microseconds of latency. I'm not disagreeing in principle with the benefits of binary encoding. But between SIMD hex codecs and hash functions, flate compression, and the broad support without any of the complexities that gRPC/protobuf ties you to as a developer, I would argue to skip the "use a good existing format" step. The binary format I already use inside the Badger database is, as I mentioned, extremely compact and includes a mechanism to string-intern the pubkeys (this is a huge saving for follow lists). It is already extremely fast, the algorithms are small and simple, and already battle-tested on WoT database usage. Signature verifications, however, are problematic. A canonical encoding used to create the signature is necessary, no matter how you wire- or disk-encode it. It's the biggest friction point for Nostr moving to a better, binary format. It's also why it was so critical to make that codec as fast as it can be. Part of the trick of the performance is that it's a subset of JSON that it encodes — only the part relevant to the exact syntax of Nostr events. It will swallow incorrectly formatted, minified JSON happily, if the extraneous characters are not expected next in the mostly-forward, no-lookahead decode process. Lookahead is the biggest reason why most JSON codecs are slow. It's also a consideration I took into account when I was pondering the idea of a human-readable but compact format that uses linebreaks instead of brackets, which saves a byte for every branch in the syntax tree.
0000 sats