Cocos Creator’s compiler has been firmly invested in the front-end technology stack from the beginning of the project. It has accumulated a lot of industry-leading low-level experience. These low-level designs support the higher-level framework and business layer development. On the one hand, they meet the needs of Cocos Creator’s own rapid iteration. On the other hand, they also support the editor implementation of Cocos’ HMI, virtual characters, and other business lines.
At this GMTC Conference, the Head of Front End for Cocos Creator vsj was invited to attend, and delivered a speech with the theme of “Cocos Creator Editor Technology Architecture and Practice” and shared the pain points and solutions encountered during our research and development, Cocos’ exploration in the field of web technology, and their views on future front-end technology trends.
Background of the project
Cocos has experienced 13 years of growth and has developed into the world’s leading digital interactive content development platform, ranking highly in open-source game engines, and has 1.6 million registered developers around the world who have made a lot of contributions to the Cocos ecosystem.
For example, users have extended many excellent plugins through Cocos Creator’s plugin system, such as the Visual Texture Editor, EasyNavMesh Pathfinder, and Grid Wayfinder.
A visual material editor is a kind of low code editor that preset some functions, connect some functions together, sets some parameters or conditions of the connection, and output resources that the engine can identify to achieve an effect.
The EasyNavMesh and NavMensh pathfinder plugins extend pathfinding functionality to game runtimes through components and scripting systems.
These are inseparable from Cocos Creator’s plugin system. It can not only support editing operations within the editor but also directly extend the game’s content. Cocos Creator is based on the front-end technology stack to open these capabilities to developers easily.
Reasons to Choose Electron
Among the many front-end technology stacks, Electron has a lot to offer. It is embedded in the Web Runtime and can complete the use of Web functions. Some complex interface operations on the web can be more convenient to achieve. If not, you can also use Canvas to accomplish some special or complex interactive operations.
At the same time, Electron uses NodeJS as a development language, which can bring us convenient module management and dynamic insert and unload of scripts.
In addition, it is one-time cross-platform development and multi-release. After the one-time development is completed, it can be released simultaneously on both Mac and Windows platforms.
At the same time, it also handles the adaptation of many functions and platform differences. For example, adapt to different scaling resolutions of the screen.
Most importantly brings us the NodeJS ecosystem, which accelerates business development and lets us focus more on the business. Once the product proposes a business requirement, we can find the corresponding solution or tool in the NPM market and quickly iterate the function so that we have time to optimize it.
Web Application Desktop
Using Electron or JavaScript or TypeScript to develop a large application, the most serious problem encountered is not the technical difficulties but the difficulty of engineering management. The freedom of JS and TS brings various uncertainties and problems to the entire project after the project is complicated.
High volume of business and related solutions
We split these problems into two directions, the first is the high complexity of the code, and the second is the remote collaboration or multi-person collaboration.
The total amount of code in the editor (including the engine) is about 1.23 million lines. This magnitude is quite frightening for a typical web application or page. Cocos has a lot of business, with more than 60 functional plugins. We support 15 platforms at the same time, all of which need to be adapted.
The second problem is collaboration. Editor developers are all over Beijing, Shanghai, Chengdu, Shenzhen, and Xiamen.
To cope with the difficulties in engineering, Cocos Creator uses a layered and modular design concept, as shown below, divided into three layers.
The top layer of the editor is centered on plugins. In Cocos Creator, everything is a plugin. Except for the low-level API of the compiler and some of the modules we encapsulate as NPM, all the functionality you see is encapsulated as plugins. The plugin system is a very important part of the editor. Not only the plugin provided by the developer but all the editor modules are also plugins.
The Creator plugin’s operating structure refers to Electron’s process division.
In writing, the Cocos Creator plugin is basically the same as an NPM module, with a package.json describing the basic information for the current plugin. Main in JSON is the primary logical entry for the plugin. After the plugin starts, it will always run like a background service.
Panels are the logical entry point; a plugin can register multiple panels. The panel can be understood as a display module on the page. It also can be closed. Contributions are one of the core mechanisms of plugins, bringing the ability to interact with each other.
Interaction between businesses
The first problem Cocos Creator has to solve after using plugins for business isolation is the interaction between different businesses. We choose to use a communication pipeline or a built-in communication manager. For example, when an object is selected in the scene area, the hierarchy manager needs to be updated, and the content of the property checker needs to be updated.
The simplest communication is one-way communication, and it brings one-way dependence. ExtensionA depends on ExtentionB.
Project management needs to clarify the dependency relationship. But let’s start with a simple example of one-way communication.
If you update a property inspector in the scene editor, the process is as follows:
We first click a character node in the Scene Editor. After selecting the node, the scene notifies the property checker. The property checker receives the notification or a message and starts a render, rendering to a certain extent. If there is insufficient data, it actively queries the scene for additional data. After the scene returns information, the rendering is complete.
But then there is a double dependency problem. The two sides interact and consult each other. This will bring many hidden dangers to the project. Therefore, in complex remote collaboration or multi-person collaboration, this problem must be solved.
The solution is also straightforward, it is unicast, and there must be a radio. Broadcasting is a logical inversion of dependencies.
Corresponding to the above example, if you use broadcast, the entire communication flow will change, as shown in the following figure:
The whole logical link becomes - the scenario and the attribute checker depend on the message manager, there is no mutual dependency between them, and the attribute checker also depends on the scenario.
Message and IPC
Message is the communication mechanism of Cocos Creator, which is actually an extension of IPC provided by Electron.
The problem with IPC is that it is interprocess interaction, and interaction with different modules of the same process cannot be achieved. And IPC is only a one-way communication. If only one-way interaction, the entire program interaction will be challenging.
So Message mainly solved the above two problems. The first step is to carry out message forwarding after the data is compressed at the time of transmission, sent to the corresponding process through IPC, decoded by the corresponding process, and then distributed to the corresponding module.
The message reply is also very simple: record an id at the time of sending, and after processing, return the data and id back to the requested process via IPC, through the id to find the previously sent code or object, and then execute the following code.
The first version of Cocos Creator implemented a callback on the left. In message processing, a first parameter is an event with a reply on it, and it needs to manually execute a reply before it can return data. This presents a very big hidden danger. Suppose the user does not execute a reply. In that case, the sender has no chance to know that it returned, and the sender does not know whether the function actually processed has terminated.
So in Cocos Creator 3.0, since ES6 introduced promises, we were able to change all messages to promises to solve the problem of callbacks. A promise object is generated and cached in the manager when the message is sent. The receiver only needs to create an asynchronous function, which will definitely end as long as the return. After receiving the end data, the sender only needs to find the just cached promise object, execute the corresponding resolve function, and then realize a complete message communication.
Contributions
The contributions mechanism is also a core mechanism in Cocos Creator.
In developing a business, Cocos Creator often needs to add business at random or unpredictable time points, so we must set aside a better expansion ability.
We want all the functionality of a business to be contained in one plugin. For example, the animation comprises three parts. The first part is the panel - the area where the animation is based on interaction. The second part is the configuration that needs to add some animations to the preferences. The third part is that the property inspector needs to render some animation-related elements. Only these three parts can be put into the animation plugin in order to achieve the real plugin or modular.
Again, take the animation diagram and the property checker, where their dependencies are the same as in the previous scenario. The animation graph depends on the message manager, and the property checker depends on the message manager and the animation graph plugin. The problem is that when developing the property manager, you didn’t know that future animations would need to be displayed in the property inspector. When a change or feature iteration is made on the animation plugin, the property inspector, as the dependent party, must also make a change or perform a functional test. This situation is particularly troublesome in the case of a large volume of business.
We want a split of the business and will rely on another inversion. After the development of the property checker, they no longer care about any business, doing a forwarding or just running the process.
To achieve dependency inversion, you give the initiative to the animation diagram, which actively registers the render type with the property inspector. After the property inspector receives it, the data is cached.
When the selected object is the registered type of animation, it will submit the rendering permission to the animation. The animation will get the data, render it according to the business, and return the rendered content to the property inspector. The property inspector will be able to complete the process. Contributions make the property checker dependent and, as a basic mechanism, no longer affected by other businesses.
The process is as follows. When the animation plugin is started, it provides the property inspector with a render type of data. After receiving the notification of the selected object, the property inspector will go to the Contributions Manager to find the corresponding data according to the type. If it finds the data, it will submit the rendering permission to the plugin that can render the data, which is the animation.
The purpose of the contributions mechanism is to allow plugins to extend each other.
The simplest example is creating a simple extension with the Contributions data in the package json with two properties, menu and message. So you can guess that this plugin registered some menus but also registered some messages for communication.
Cocos Creator isolates all functionality in plugin units through the message and Contributions mechanisms. There are also unexpected plugins, such as the menu and message within the Contributions data in the example, which are also plugins. In other competing products or compilers, menu and message is the underlying mechanism. But in Cocos Creator, everything is a plugin.
Source Protection
In addition to project management, Cocos Creator has made many attempts at source code protection. Electron uses JavaScript, which can only be compressed or confused, making the code logic complicated and ugly, but spend some thought or can find out the core running logic.
The first thing we tried on Cocos Creator was translating some JavaScript primitives into C + +. Through two ways, one is compiled into wasm, through WebAssembly. The second one is compiled to .node. This approach works for the most part but has run into some problems.
So we tried another approach - precoding bytecode. Electron runs a V8 engine. When V8 runs on JavaScript, it precompiles a piece of code into binary bytecode and finally runs the binary number when it runs. We need to compile the JavaScript code into binary bytecode and call the underlying interface to parse the code where it needs to run.
But there are also severe problems with the scheme. It strongly depends on the runtime VM environment. When the base or Electron upgrades, the original bytecode will become invalid. Even when the runtime environment changes, precompiled bytecode cannot be parsed. The most unacceptable thing for Cocos Creator is that some NodeJS global flags change to invalidate precompiled code.
So Cocos tried a third option, with a decoding tool built into the C + + layer. Please read the file from the JavaScript layer, get the file data, and send it to C + + for decryption. The advantage of this approach is that the work is very small, as long as you find anywhere in the C + + layer to put the code. But for Electron, the C + + layer is the source code of Electron or NodeJS. These are intrusive changes, and intrusive changes will certainly create some work for subsequent upgrades.
Summary and Outlook
In the process of using Electron, in addition to the project management and source code protection issues just mentioned, there are the following problems.
The first is that the front-end technology iterates too fast. Cocos Creator’s game editor’s development is very focused on ecology. We want to expose APIs as far as possible but not make them incompatible. The framework and version of the front end are frequently updated, which limits our external use of these features. For example, in the earliest days of Cocos Creator, vue 1.x was used to make UI. In some external APIs, vue objects were exposed directly to plugin developers. A few years later, vue 2.x and even vue 3.x was released, and when we wanted to upgrade, we found that we couldn’t upgrade. Because once upgraded, the original plugin is invalid. Of course, we can choose to force compatibility with the original code, another set of mechanisms, or API to use a new syntax and framework, but this will bring more significant learning costs and maintenance burden. So we can use the various frameworks at the front end but don’t usually provide them with them.
The second is the low performance of the JavaScript language. It can also be solved through the function of the original biochemical part of JavaScript.
Next comes Privilege Management, which refers to which parts of a file or network API a piece of code can access. This part of NodeJS is challenging to do. We can simulate some privilege management by hijacking the API.
Finally, it isn’t easy to expand the platform’s capabilities. Electron gives us a lot of platform capabilities. Still, if there is a demand for more than these capabilities, you can only be forced to modify the Electron source code.
Finally, a look at the future of the web. Front-end technology has been applied in a wide range of areas, and these applications can probably be divided into two categories. The first category is embedded WebRuntime scenarios, representing Electron, similar also have tauri, and so on. The second is the way in which technical references learn from each other, such as Flutter, which draws on some of the models of CSS, elastic layouts, and so on.
The embedded Runtime can be understood as allowing developers to enter the front-end ecology and environment. When Runtime capabilities and performance are sufficient, I believe many apps will be willing to migrate to the web ecosystem.
Technology reference and integration is the front-end technology to go outward to spread their influence. For example, games, car machines, or VR may not run Web Runtime. They may be self-developed, or they may be mature layout frameworks. What they have in common is that they can absorb some mature ideas or ideas from the web. This approach is precisely a way to derive their own influence outwards, which may affect the basic development direction of these emerging fields.