0–1 Create a modern drawing framework

Our company has decided on mind mapping at the beginning of this year (2022). After more than half a year of research and practice, we have initially completed a mind map component based on a self-developed drawing framework and successfully integrated it into our PingCode Wiki. This article mainly Let’s discuss some design points of this drawing framework (Plait) and some key technologies for the implementation of mind mapping.

Yes, this article was written in 2022

Introduction

We did a lot of research on mind maps and flow charts in the early stage. For flow charts, we studied excalidraw and react-flow. They are both libraries based on the react framework and have a high reputation in the community. For mind maps, we have studied mind-elixir and mindmap-layouts (automatic layout algorithms). The development of mind maps in the open source field is not very good, and there are no mature and well-known works.
Although our early goal is to develop “mind maps”, our final product goal should be to create an “integrated interactive drawing board” that includes mind maps, flow charts, free brushes, etc. However, the open source community currently lacks such an integrated drawing framework to implement an integrated interactive drawing editor that integrates mind maps, flow charts, and free brushes. Therefore, we combined our experience in making editor frameworks to re- Architected a drawing framework (Plait), with a plugin mechanism, and based on it, the mind map plugin was implemented and implemented into the PingCode Wiki product.

mind-exlixir introduction:

Advantages:

Although small like a sparrow, it is fully equipped.
It’s a pure JS library, lightweight in nature.

Disadvantages:

It doesn’t depend on front-end frameworks, and its development approach differs from mainstream methods.
The architecture design doesn’t have many merits, and the node layout method is not easily expandable.

Let’s officially delve into today’s topic, which is divided into four parts:

1. Design of Drawing Framework
2. Mind Map Architecture
3. Automatic Layout Algorithm for Mind Map
4. Framework History and Future

一、Drawing frame design

This part will first briefly introduce the considerations for selecting a drawing solution (Canvas vs SVG), then introduce the design of the core parts of the Plait drawing framework: plugin mechanism, data management, and finally introduce the advantages of the framework.

Drawing scheme: Canvas vs SVG

Canvas or SVG. In fact, there is no clear answer in the community. We refer to the solution selection of some well-known products. Both SVG and Canvas are available and the implementation effects are not bad. For example, Yuque’s whiteboard uses SVG, and ProcessOn uses It’s Canvas, Excalidraw uses Canvas, drawio uses SVG and so on. Because we have no experience using Canvas, and our mind map nodes hope to support rich text content, we temporarily selected SVG, which is more friendly to the DOM, and thought we would try this solution first.
After such a long period of verification, we found that the SVG-based solution has no obvious shortcomings. We have also verified the performance issues. There is no problem with rendering of mind map nodes that support 1000+, and the operation is still very smooth.

For SVG drawing, we do not directly use the underlying API of SVG, but use a third-party drawing library roughjs.

Let’s take a look at the “plug-in mechanism” part of the Plait framework. This part is inspired by the rich text editor framework Slate.

Plugin mechanism

There are many business directions that can be developed in depth in the field of web front-end drawing. How to develop functions in different business directions based on the same framework requires the use of a plug-in mechanism.
The plug-in mechanism is an important feature of the Plait framework. The bottom layer of the framework does not provide specific business function implementation. Business functions need to be implemented based on the plug-in mechanism, as shown in the following figure:

In layman’s terms, the plugin mechanism is a basic bridge built at the framework level, providing necessary support for implementing specific business functions. The Plait plugin mechanism consists of three core elements:

1. Abstract Data Paradigm (Plugin Data)
2. Overwriable Behavior (Recognizing Interactions)
3. Overwriable Rendering (Controlling Rendering)

The core of this part is to design rewriteable methods. Currently, there are two main categories in Plait:
The first category is used to implement custom interactions: mousedown, mouseup, mousemove, keydow, keyup
The second category is used to implement custom rendering: drawElement, redrawElement, destroyElement
Then there is the design of the connection between the framework layer and the plug-in. This part is currently designed relatively loosely in plait/core. drawElement can return a DOM element of the SVGGElement type or a frame component, which can be directly connected to the frame or Can be connected based on DOM.
At present, the entire Plait framework is implemented based on the Angular framework. In the future, design patterns that break away from the framework may be considered. This is not the focus of this article.

For example: Three steps to draw a circle plugin

Step 1: Define the data structure

export interface CircleElement {
    type: 'cirle';
    radius: number;
    dot: [x: number, y: number];
}

Step 2: Process the circle drawing interaction

board.mousedown = (event: MouseEvent) => {
  if (board.cursor === 'circle') {
    start = toPoint(event.x, event.y, board.host);
    return;
  }
  mousedown(event);
};
board.mousemove = (event: MouseEvent) => {
  if (start) {
    end = toPoint(event.x, event.y, board.host);
    if (board.cursor === 'circle') {
      // fake draw circle
    }
    return;
  }
  mousemove(event);
};
board.globalMouseup = (event: MouseEvent) => {
  globalMouseup(event);
  if (start) {
    end = toPoint(event.x, event.y, board.host);
    const radius = Math.hypot(end[0] - start[0], end[1] - start[1]);
    const circleElement = { type: 'circle', dot: start, radius };
    Transforms.insertNode(board, circleElement, [0]);
  }
};

Step 3: Implement the circle drawing method

board.drawElement = (context) => {
  if (context.elementInstance.element.type === 'circle') {
    const roughSVG = HOST_TO_ROUGH_SVG.get(board.host) as RoughSVG;
    const circle = context.elementInstance.element as unknown as CircleElement;
    roughSVG.circle(circle.dot[0], circle.dot[1], circle.radius);
  }
  return drawElement(context);
}

This is the simplest plug-in example, which implements a circle drawing plug-in through the bridge provided by the framework: drag and drop to draw a circle -> generate corresponding circle data -> render the circle based on the data.

The plug-in mechanism probably consists of these contents. Let’s take a look at the data management part.

Data management
Data management is a very important part of the Plait framework. It is the soul of the framework. The previous plug-in mechanism is an external manifestation and mainly includes the following features:

1. Provide basic data model
2. Provides atomic-based data change methods (Transforms)
3. Based on immutable data model (based on Immer)
4. Provides a Change mechanism to cooperate with the framework to complete data-driven rendering
5. Integrated with the plug-in mechanism, the data modification process can be intercepted and processed

These are very excellent features, which can not only complete data constraints, but also flexibly implement many complex and advanced requirements. I feel that the design and implementation of this area can actually be regarded as a kind of state management for specific scenarios.

Framework status flow diagram:

The closed loop of the plug-in mechanism mentioned above depends on the data model state as the standard. The final closed loop of the plug-in is shown in the figure below:

framing effect

1. Implement layered business processing based on the plugin mechanism
2. Abstract the data model, providing unified data transformation operations
3. Standardize the data flow

The greatest significance of the framework is to deconstruct in layers to reduce the complexity of the business. Each field or module only handles its own affairs.

For example, front-end frameworks and component-based development are able to group certain logic into a logical domain (component), instead of mixing everything together. This is the trend of architecture evolution.

Architecture diagram:

二、Mind Map Architecture

Here is a brief introduction to the overall technical solution of the mind map plugin, but it will not be introduced in detail because it is implemented based on the Plait framework. The large solution must be consistent with the Plait framework.

1. Overall solution

Our overall solution is SVG + Richtext.

Drawing uses SVG. Currently, our mind nodes, node connections, expand and collapse icons, etc. are all drawn based on SVG.

Node content is embedded into SVG using foreignObject inclusion, a scheme that enables node content to support rich text.

2. Functional solution

The core interactive processing and rendering of mind are all done in overridable methods and integrated with Plait.

The mind plugin is only responsible for the rendering of the mind part. As for the rendering of the entire canvas and the movement, zooming and zooming of the canvas, etc., they are the underlying functions of the framework.

Because Plait supports extending any element and rendering multiple elements, it is only determined by data, so it supports rendering multiple brain maps at the same time.

The underlying mind plugin does not include toolbar implementation, it only handles core interaction, rendering, and layout.

3. Componentization

The overall mechanism of mind component rendering and node rendering is what we often mention on the front end: componentization and data-driven. Although node rendering within components still creates DOM and destroys DOM, large functions are still divided through components.

There are two very important components in the mind map business: MindComponent and MindNodeComponent. MindComponent handles the overall logic of the mind map, such as executing the node layout algorithm, and MindNodeComponent handles the logic of a certain node, such as node drawing, connection drawing, and node theme drawing. wait.

The reason why I bring this part up for discussion is because I think this idea is actually a continuation of the mainstream front-end framework ideas, including being unified with the overall mechanism of the Plait framework.

4. Drawing editor

This can be understood as the encapsulation of the business layer. The business layer decides which extension plugins to integrate and further expands the upper-layer functions of the plugins (such as the implementation of the mind node toolbar). The Mind plugin layer does not depend on our component library and business components, so the toolbar Such scenarios that require component library components are implemented in the business layer, so that the mind plugin layer can reduce dependencies and maintain focus.

三、Automatic layout algorithm

Automatic node layout is a core technology of mind maps. It is a decisive factor in the beauty of mind maps and the expressiveness of content. It focuses on how nodes are distributed. This part is neither complex nor simple, and it includes the following parts. :

1. Layout classification
2. Node abstraction
3. algorithm process
4. direction change
5. Layout nesting

Layout classification

Introduce the layout classification of mind maps:

Aesthetic standards

As mentioned before, mind maps have high requirements for the display of visual trees, and they need to be beautiful (this is very intuitive, everyone’s aesthetic may be different, but it should also have some basic standards), so it needs basic Aesthetic standards:

1. Nodes do not overlap
2. Child nodes are arranged in the specified order
3. The parent node is in the center of the child node (logical layout)
4. Nodes at different levels do not overlap in the direction of the main axis

Node abstraction

In order to simplify the drawing of visual trees, [Reingold-Tilford] proposed that the spacing between nodes and the drawing connections can be abstracted. Add spacing between nodes by adding gaps to their width and height. As shown in the figure below, the solid line box shows the original width and height, and the dotted line box shows the width and height after adding gaps.

Our mind map automatic layout follows this abstraction:

1. Node layout itself does not pay attention to node connections, only the width, height and spacing of nodes.
2. The nodes are abstracted from the spacing (node width, height and node gap are abstracted into a virtual node LayoutNode)
3. The layout algorithm is based on LayoutNode for layout

When the node is laid out, its width and height have been integrated with the actual width and height and the upper, lower, left, and right gaps, which can reduce the complexity of automatic layout. The above picture is actually the result of our layout. After the node is abstracted from the spacing, the node The vertical top position of is the bottom coordinate of its parent node, and the bottom coordinate of the parent node is its top coordinate plus its height. The logical relationship between real nodes and virtual nodes is as shown in the figure below:

Algorithm execution process

Algorithm flowchart:

Using an example containing three nodes to introduce its automatic layout process, the ideal result should be as shown below:

Step 1. Pre-operation: Construct LayoutNode

This is the node abstraction mentioned earlier. It is an abstract node used to construct the layout based on the node width and height and the gap between nodes. At this time, the three are in the initial state, and the x and y coordinates are zero and overlap each other, as shown in the following figure:

The left side is the real state, the dotted box on the right side has no special meaning, it is just a non-overlapping indication.

Step 2. Layout preparation: vertical separation

Layering is based on the parent-child relationship of nodes, ensuring that parent-child nodes do not overlap in the vertical direction (node 0 does not overlap with nodes 1 and 2).

Step 3: Separate sibling nodes

It is to separate “Node 1” and “Node 2” to ensure that they do not overlap horizontally.

Step 4: Locate the parent node

Based on “Node 1” and “Node 2”, reset the horizontal position of the parent node “Node 0” to ensure that the parent node is centered horizontally with “Node 1” and “Node 2”.

Layout results:

The above is a complete layout process (logical layout). The logic is not complicated. Even if there are more levels and nodes, you only need to recursively execute “Step 3” and “Step 4”.

It can be seen that “logical layout” is completed using only the first four steps in the “algorithm flow chart”. The last step “direction change” is to achieve “logical transformation” through mathematical transformation on the basis of “logical layout”. “Logical right” and other layouts are briefly explained in the following section.

direction change

1. Logically -> Logically

It can be seen that this is a transformation relationship in the vertical direction. They should be symmetrical based on a horizontal line. The specific transformation relationship is as shown in the figure below:

You can see that the “y point” of the rightmost and bottom node should correspond to the “y point” of the rightmost and top node. Their positional relationship should be: y= y — (y-yBase) * 2 — node.height.

Note that the up-down transformation should only involve displacement, not up-down flipping, that is, the direction inside the node remains unchanged, and y corresponds to y`, which correspond to the points on the lower edge of the node.

2. Logic down -> Logic right

As can be seen from the above figure, this logical transformation is not complicated: it is a vertical to horizontal transformation process, which reflects the transformation of the x, y directions and node width and height in the layout algorithm layer, such as:

1. Vertical layering: vertical layering needs to be transformed into horizontal layering
2. Add buildTee process: Layered-based nodes need to convert node width to height and x coordinate to y coordinate

Finally, in “Direction Transformation”, transform the width, height, x, and y back:

Get layout results:

3. Logical right -> logical left

The position correspondence relationship from logical right to logical left should be similar to the above-mentioned logical down to logical relationship, so I won’t go into details here.

There are roughly three types of direction changes. The following is an introduction to the idea of layout nesting.

Layout nesting

Let’s first look at a diagram of layout nesting:

The second child node in the above picture uses another layout (indented layout), which is layout nesting. Layout nesting still needs to ensure the “aesthetic standards” mentioned above, such as nodes not overlapping, parent nodes aligned in the center, etc. .

Simple thinking: The subtree with independent layout in the layout nest has an impact on the overall layout because its extension direction is uncontrolled. However, if the subtree with independent layout is regarded as a whole, the subtree can be calculated in advance. The layout of the tree, and then substituting the subtree as a whole into the layout algorithm can shield the impact of the subtree extension method on the overall layout.

The overall processing idea is shown in the figure below:

There can be an abstraction here: abstract the child nodes with independent layout into a black box (I call it BlackNode), then the influence of the subtree layout will be brought into the main layout, and the subtree layout can be maintained independence.

Key point: The layout of the subtree with independent layout needs to be calculated first, and then the parent node layout can be calculated

4. Framework History/Future

It took about a year from technical research to architecture conception to implementation of the architecture into products. The core work will be concentrated from January to September 2022. The approximate timeline is as follows:

Some ideas for the future of Plait framework

四、Conclusion

This article mainly introduces some technical solutions for making drawing applications from scratch, self-developed drawing frameworks, and implementation of mind mapping scenarios. As a Web front-end developer, I feel very lucky to have the opportunity to have such a thing. There are still many things for the Plait framework in the future. I hope it can develop into a mature open source community work, and I also hope that students who are interested in the drawing framework can join the open source construction of Plait.

中文版本：https://zhuanlan.zhihu.com/p/609592474

Welcome to star：https://github.com/worktile/plait