Context
At GOPAS, I worked on a new generation of StoryBoard, a large WPF/MVVM application built on .NET Framework.
The project was originally started by my colleague, who created the initial architecture, the capture core, and the first database model. He then handed the application over to me, and from that point I worked on its development independently.
At that time, the application was still missing most of its functional parts: the user interface, application logic, work with lessons, data mapping, the editor for captured elements, filtering, and the completed export process.
My task was to turn the existing technical foundation into an application that internal users could actually use. I implemented the desktop UI and application logic, extended the database model, and added mapping between Entity Framework entities, domain objects, and view models. I also worked on the lesson editor, export, ClickOnce deployment, and fixes in the capture core.
When designing features, I worked mainly with internal users of the previous StoryBoard version. They knew the process of creating training lessons and helped me understand how the resulting application needed to work in practice.
How StoryBoard worked
StoryBoard was not just a regular editor. Its main job was to capture the look and behavior of another application and prepare data for an interactive software tutorial.
The user usually worked with two monitors. On one, they had StoryBoard open. On the other, they had Microsoft Word, Excel, or another application for which they were creating a lesson.
In StoryBoard, the user started capturing the current state of the application. The capture core first created a screenshot of the window and then used Windows UI Automation to walk through its structure. It gradually visited individual buttons, menus, and other controls, recording their appearance in the normal state and on hover.
Capturing also included moving across the screen with the cursor. StoryBoard used this to detect areas where the cursor changed appearance, for example from the standard arrow to a text cursor or another pointer type. It also captured tooltips and other visual changes that could not be obtained from a static screenshot alone.
If a tutorial was meant to show writing text and then formatting it, the user first let StoryBoard capture Word in its initial state. Then they wrote the text, captured the current state of the application again, and repeated the same process after changing the text to bold, italic, or after adjusting the font.
Each lesson contained several windows. One window represented one capture of the application at a specific point in the workflow and contained a screenshot, the structure of UI elements, tooltips, areas with different cursor shapes, and other captured data.
After completion, the lesson was exported. The resulting data was then used by a downstream application that played the interactive software tutorial.
Because one window could contain dozens of UI elements and capturing each of them took several seconds, StoryBoard used a local cache. The capture core compared identifiers of found elements with already stored records and did not have to capture known elements again. This significantly sped up repeated captures of the same application in later parts of the lesson.
Automatic capture was not perfect in every situation, so StoryBoard also included an editor. Users could fix the position or appearance of an element, add a missing capture, or manually adjust the result of automatic capture.
The captured image could also be opened in a tool such as Photoshop. StoryBoard watched for the external edit to finish and, after the changes were saved, reloaded the modified image and updated the relevant element in the lesson.
What the application supported
StoryBoard supported, for example:
- creating training lessons composed of several windows and steps;
- capturing applications, their windows, and individual UI elements;
- capturing normal element appearance, hover states, and tooltips;
- mapping areas with different cursor types;
- managing scenes, instructions, texts, languages, metadata, and source files;
- validating lesson structure and content;
- manually correcting automatically captured elements;
- saving and reopening unfinished lessons;
- exporting to HTML, JSON, XML, and DOCX;
- packaging the resulting lesson and its resources into an archive.
Technical context
StoryBoard consisted of several C# projects with separated responsibilities. The solution contained WPF views, MVVM view models, application services, Entity Framework entities, repositories, mappers, the export module, screen capture components, shared utilities, and automated tests.
The project used Ninject for dependency injection, AutoMapper for mapping between data representations, and Entity Framework 6 over SQL Server.
The architecture was designed to guide less experienced developers in a safe and predictable way. Because of that, it contained a larger number of intermediate layers, abstractions, and rules intended to limit incorrect use of individual parts of the system.
During practical development, however, I gradually found that some of these abstractions made normal changes more complicated and sometimes conflicted with the natural way MVVM works. I therefore simplified the architecture, removed unnecessary connections, and adjusted its parts so they better matched the real needs of the application.
The database model was a solid foundation, but as new features were added, I had to extend it further and gradually refine it.
One important change was separating Entity Framework entities from the presentation layer. I added domain objects and mappers between database entities, the domain model, and view models, so the user interface would not be directly dependent on the database structure.
Editor and working with elements
In the editor, I implemented work with captured elements and their visual representation.
Elements could be moved using drag and drop instead of manually entering coordinates. The visual canvas was also synchronized with the element tree: selecting an item on the canvas selected the corresponding item in the tree, and selecting an item in the tree highlighted the element in the editor.
That allowed users to correct inaccuracies in automatic capture more quickly and add parts that could not be captured automatically.
Screen capture and Windows API
One of the most technically interesting areas was capturing the screen, application windows, and individual UI elements.
The capture core used Windows UI Automation, User32, GDI32, mouse hooks, and functions such as Graphics.CopyFromScreen, GetWindowDC, CreateCompatibleBitmap, and BitBlt.
I did not build the core from scratch, but as the application was used in practice, I had to gradually understand it and fix its behavior. Many problems were not related to ordinary application logic, but to the real behavior of Windows, application windows, the cursor, focus, tooltips, and different states of captured programs.
Support for newer Microsoft Office versions
During the first practical use of the capture core, we discovered that it did not work correctly with the then-current versions of Word, Excel, and PowerPoint.
While refactoring, I found that the capture tool used a hard-coded list of elements to ignore. Newer Office applications contained overlay elements covering almost the entire window, and because of them the capture tool could not reach the elements underneath.
I separated filtering from the capture core itself and gradually turned it into a configurable system. The user could target a problematic element and add it to the ignored elements. The configuration was first stored locally, and later we extended the solution with a central database of verified elements shared between users.
This not only restored support for newer Office versions, but also created a solution that could be continuously adapted to other applications and their new versions.
Export pipeline
The export part was based on the older version of StoryBoard, but it had to be substantially reworked for the new application.
Its job was to transform the internal lesson structure into formats such as HTML, JSON, XML, and DOCX, attach images and other resources, and preserve the exact format expected by downstream parts of the system.
The original implementation was mostly a long linear process with repeated logic. During the rework, I gradually separated individual parts into smaller named methods, removed duplication, and made the processing of scenes, elements, colors, and other data easier to understand.
At the same time, backward compatibility with the existing import and downstream tools had to be preserved. Even seemingly unimportant differences, such as the format of hexadecimal colors in different parts of the JSON output, could make lessons impossible to load again.
It was one of my first significant experiences with refactoring legacy code. It was not enough to create a cleaner implementation. It was just as important to recognize and preserve hidden contracts that other parts of the system depended on.
Testability
Testability was an important part of StoryBoard’s architecture. The project contained tests for the data layer, DTOs, view models, and broader application behavior.
It used test replacements for the database and file system, the Moq library, and facades around I/O operations. Thanks to that, it was possible to test application logic without a direct dependency on a real database, file system, or specific environment.
This experience showed me that testability does not appear only when tests are being written. It has to be reflected in the design of the application itself: in the way we separate the database, file system, mapping, application services, and user interface.
What I took from the project
StoryBoard was one of the first projects where I carried responsibility for the application as a whole, not only for one technical layer.
I worked across desktop UI, application and data logic, the database model, architecture, export, deployment, and integration with the operating system and external applications.
At the same time, I learned that working on an existing system is not just a matter of writing better code. Every change has to respect historical behavior, existing data, hidden dependencies, and the way real users work with the application.
StoryBoard was one of my first experiences where I began to see software as a whole product: a combination of code, data, user workflows, historical constraints, and operational reality.
Summary
I took over StoryBoard when the basic architecture, capture core, and initial database model already existed. I then independently implemented most of the functional part of the application: user interface, application logic, data mapping, lesson editor, work with UI elements, database model extensions, export pipeline, and ClickOnce deployment.
One of my most significant technical contributions was solving compatibility between the capture core and newer Microsoft Office versions, and turning hard-coded filtering into a configurable system with local profiles and a central database of shared rules.
During the project, I moved from implementing individual features to taking responsibility for the functioning of the whole application and its practical use.