Is there a tool to generate binary protocol figures out of a spec?

Question

I need to document quite many different binary formats and would like to present them with a figure visualizing the different fields, their length and names.Similar to how internet protocols are presented in RFC documents, but rather not in text mode. Preferably in some vector format, like SVG, to make it easy to embed these figures in different kind of documents.Up to now I have mainly used Microsoft Visio or even Excel for this. This is very tedious and ineffective. I think there must exist a tool for this where you just define the protocol (e.g. using Kaitai Struct syntax or something simlar) and then generate the protocol figure.So far I have only found tools for generating such figures in text format.Any suggestions?

curiousfab · Accepted Answer

I used Wavedrom to generate beautiful SVGs for a similar purpose in the past. Maybe it is useful for you, too:https://github.com/wavedrom/wavedrom

zelphirkalt · Answer

Does this help with documentation and automatically create diagrams: https://github.com/gsingh93/ob-bitfield ? Was recently posted here on HN. Org format is a powerful format and it can be exported to many other formats using Emacs or maybe Pandoc.

contingencies · Answer

I always appreciated the clarity of diagrams in TCP/IP Illustrated Volume 1: The Protocols.[0] IIRC the diagrams were generated with graphviz using dot, which can output SVG. You can trivially post-process the SVG to alter fonts or other display related attributes as necessary.
In terms of data sources, you can potentially parse the metadata for known protocols out of a good protocol dissector like tcpdump[1] or Wireshark[2], or https://www.luismg.com/protocol/
See https://graphviz.readthedocs.io/en/stable/examples.html#stru... for a tabular example.
[0] A decade or more ago someone posted the source TeX. Unfortunately I lost it. If anyone has it, sharing is caring. [1] https://github.com/the-tcpdump-group/tcpdump [2] https://github.com/wireshark/wireshark/tree/master/epan/diss...

OJFord · Answer

Recently - 'Why isn't there an OpenAPI for binary formats': https://news.ycombinator.com/item?id=30806032You might find something suitable there, from what I recall there was some disagreement about whether there is or not depending on what the scope is of what you want to achieve.

renewiltord · Answer

You can&rsquo;t specify the width of a single bit in the diagram but I found https://www.luismg.com/protocol/ otherwise nice.It&rsquo;s ASCII though.

Shastick · Answer

IIRC, thrift (https://thrift.apache.org/) could render defined types/structs to an SVG.
Edited to add: it’s the graphviz generator -> https://thrift.apache.org/tutorial/graphviz.html
I’d guess gRPC and other such IDLs have a way to render similar things as well.

sennight · Answer

This isn't used for bitfields (er, shouldn't really be), but you might find it useful for less granular parsing documentation:https://en.wikipedia.org/wiki/Railroad_Diagrams

kuroguro · Answer

I think Kaitai had an export feature trough GraphViz (haven't tried it tho)
_edit_
Yeah, there's some samples on their page: https://formats.kaitai.io/rar/rar.svg
Not sure if that's what you're looking for?

sgoto · Answer

I've been using SVG bob as a wasm binary. Here is a demo: https://code.sgo.to/2021/12/07/ascii-diagrams.html

tylergetsay · Answer

https://github.com/ReFirmLabs/binwalk might help, it won't generate the visuals but can give you offsets

rurban · Answer

extraction not imho.but the usual tools are pic (for diagrams) and grap (for graphs), as linked from here: http://n-t-roff.github.io/heirloom/doctools.htmlthis is the manual https://pikchr.org/home/uv/pic.pdfI've also used postscript for extreme usages, the doctools can also include ps verbatim.

memalign · Answer

Not exactly what you&rsquo;re looking for (this doesn&rsquo;t generate an image from a spec/protocol), but I created a simple image encoding in ASCII and editor that generates PNG (choose &ldquo;Still&rdquo; instead of &ldquo;Wobble&rdquo;):https://memalign.github.io/m/pceimage/index.htmlI made this so I can embed images in code.

RonaldOlzheim · Answer

Can you send me a linkedin request to talk about it?

sgtnoodle · Answer

Maybe look into mermaid? It has like a dozen different modes.

Is there a tool to generate binary protocol figures out of a spec?

I used Wavedrom to generate beautiful SVGs for a similar purpose in the past. Maybe it is useful for you, too:
https://github.com/wavedrom/wavedrom

Does this help with documentation and automatically create diagrams: https://github.com/gsingh93/ob-bitfield ? Was recently posted here on HN. Org format is a powerful format and it can be exported to many other formats using Emacs or maybe Pandoc.

Recently - 'Why isn't there an OpenAPI for binary formats': https://news.ycombinator.com/item?id=30806032
You might find something suitable there, from what I recall there was some disagreement about whether there is or not depending on what the scope is of what you want to achieve.

You can’t specify the width of a single bit in the diagram but I found https://www.luismg.com/protocol/ otherwise nice.
It’s ASCII though.

IIRC, thrift (https://thrift.apache.org/) could render defined types/structs to an SVG.
Edited to add: it’s the graphviz generator -> https://thrift.apache.org/tutorial/graphviz.html
I’d guess gRPC and other such IDLs have a way to render similar things as well.

This isn't used for bitfields (er, shouldn't really be), but you might find it useful for less granular parsing documentation:
https://en.wikipedia.org/wiki/Railroad_Diagrams

I think Kaitai had an export feature trough GraphViz (haven't tried it tho)
_edit_
Yeah, there's some samples on their page: https://formats.kaitai.io/rar/rar.svg
Not sure if that's what you're looking for?

I've been using SVG bob as a wasm binary. Here is a demo: https://code.sgo.to/2021/12/07/ascii-diagrams.html

https://github.com/ReFirmLabs/binwalk might help, it won't generate the visuals but can give you offsets

extraction not imho.
but the usual tools are pic (for diagrams) and grap (for graphs), as linked from here: http://n-t-roff.github.io/heirloom/doctools.html
this is the manual https://pikchr.org/home/uv/pic.pdf
I've also used postscript for extreme usages, the doctools can also include ps verbatim.

Can you send me a linkedin request to talk about it?

Maybe look into mermaid? It has like a dozen different modes.

Is there a tool to generate binary protocol figures out of a spec?

I used Wavedrom to generate beautiful SVGs for a similar purpose in the past. Maybe it is useful for you, too:https://github.com/wavedrom/wavedrom

Does this help with documentation and automatically create diagrams: https://github.com/gsingh93/ob-bitfield ? Was recently posted here on HN. Org format is a powerful format and it can be exported to many other formats using Emacs or maybe Pandoc.

Recently - 'Why isn't there an OpenAPI for binary formats': https://news.ycombinator.com/item?id=30806032You might find something suitable there, from what I recall there was some disagreement about whether there is or not depending on what the scope is of what you want to achieve.

You can’t specify the width of a single bit in the diagram but I found https://www.luismg.com/protocol/ otherwise nice.It’s ASCII though.

IIRC, thrift (https://thrift.apache.org/) could render defined types/structs to an SVG.Edited to add: it’s the graphviz generator -> https://thrift.apache.org/tutorial/graphviz.htmlI’d guess gRPC and other such IDLs have a way to render similar things as well.

This isn't used for bitfields (er, shouldn't really be), but you might find it useful for less granular parsing documentation:https://en.wikipedia.org/wiki/Railroad_Diagrams

I think Kaitai had an export feature trough GraphViz (haven't tried it tho)_edit_Yeah, there's some samples on their page: https://formats.kaitai.io/rar/rar.svgNot sure if that's what you're looking for?

I've been using SVG bob as a wasm binary. Here is a demo: https://code.sgo.to/2021/12/07/ascii-diagrams.html

https://github.com/ReFirmLabs/binwalk might help, it won't generate the visuals but can give you offsets

extraction not imho.but the usual tools are pic (for diagrams) and grap (for graphs), as linked from here: http://n-t-roff.github.io/heirloom/doctools.htmlthis is the manual https://pikchr.org/home/uv/pic.pdfI've also used postscript for extreme usages, the doctools can also include ps verbatim.

Can you send me a linkedin request to talk about it?

Maybe look into mermaid? It has like a dozen different modes.

I used Wavedrom to generate beautiful SVGs for a similar purpose in the past. Maybe it is useful for you, too:
https://github.com/wavedrom/wavedrom

Recently - 'Why isn't there an OpenAPI for binary formats': https://news.ycombinator.com/item?id=30806032
You might find something suitable there, from what I recall there was some disagreement about whether there is or not depending on what the scope is of what you want to achieve.

You can’t specify the width of a single bit in the diagram but I found https://www.luismg.com/protocol/ otherwise nice.
It’s ASCII though.

IIRC, thrift (https://thrift.apache.org/) could render defined types/structs to an SVG.
Edited to add: it’s the graphviz generator -> https://thrift.apache.org/tutorial/graphviz.html
I’d guess gRPC and other such IDLs have a way to render similar things as well.

This isn't used for bitfields (er, shouldn't really be), but you might find it useful for less granular parsing documentation:
https://en.wikipedia.org/wiki/Railroad_Diagrams

I think Kaitai had an export feature trough GraphViz (haven't tried it tho)
_edit_
Yeah, there's some samples on their page: https://formats.kaitai.io/rar/rar.svg
Not sure if that's what you're looking for?

extraction not imho.
but the usual tools are pic (for diagrams) and grap (for graphs), as linked from here: http://n-t-roff.github.io/heirloom/doctools.html
this is the manual https://pikchr.org/home/uv/pic.pdf
I've also used postscript for extreme usages, the doctools can also include ps verbatim.