An evidence-based guidance framework for neural network system diagrams.

Accurate communication of research is essential. We present the first evidence-based framework for formatting neural network architecture diagrams within scholarly publications. Neural networks are a prevalent and important machine learning component, and their application is leading to significant scientific progress in many domains. Diagrams are key to their communication, appearing in almost all papers describing novel systems. However, there are currently no established, evidenced-based conventions describing how they should be presented. We study the use of neural network system diagrams through interviews, card sorting, and qualitative feedback structured around ecologically-derived example diagrams. We find that diagrams in scholarly publications can be difficult to interpret due to ambiguity and variance in their presentation, and that there is a high diversity of usage, perception and preference in both the creation and interpretation of diagrams. We examine the results in the context of existing design, information visualisation, and user experience guidelines and use this foundation to derive a framework for formatting diagrams, which is evaluated through an experimental study, and a comprehensive "corpus-based" approach examining properties of published diagrams in top neural network venues. The studies demonstrate that 1) both the usability and utility of the framework are high and 2) papers containing diagrams that conform to the guidelines receive more citations than those containing diagrams that violate them.