The Future of Terraform Visualizations
When I set out to write Terramaid, a tool that transforms Terraform configurations into visualizations using Mermaid diagrams, I didn't fully grasp the niche problem I aimed to tackle or the obstacles ahead. My goals were to improve my Go skills, contribute to the cloud tooling ecosystem, and have fun. The weekend after releasing the tool, I was amazed by the overwhelming support, inquiries, and feature requests. Although I have contributed various tools and knowledge bases to the cybersecurity community over the years, the support for Terramaid was unmatched. I began to think I was really onto something. However, I have always been more of an engineer than a visionary. To paraphrase Linus Torvalds, I focus on fixing the potholes in front of me rather than dreaming of going to the moon. Developing Terramaid has forced me to step back, look at the bigger picture of how we can drive the next generation of Terraform visualizations, and start building.
The Problem
In my mind, Terramaid has one mission: to create good visualizations of Terraform configurations so engineers can easily see what will be deployed into their environments. The challenge is that many factors contribute to making something good. In the past, I used tools like Rover to visualize Terraform plans. While Rover is comprehensive, well-documented, supported, and easy to use, it has downsides. It doesn't seamlessly integrate into pipelines where I needed visualizations the most, and a large number of resources quickly turned into a tangled web in the diagram, making it less valuable for understanding deployments. With this knowledge, my list of requirements grew. I wanted to develop a tool that creates good visualizations by minimizing unnecessary output in the diagrams, providing a customizable interface to meet engineers' needs, integrating easily into existing CI/CD pipelines while also being able to run locally, and offering a well-documented, community-supported tool that does its job well.
The First Iteration
The first release of Terramaid (0.1.0) was not pretty. The tooling was unreliable, inconsistent, not performant, and sloppy. It was a typical Minimum Viable Product (MVP) that felt rushed out the door. Why I rushed a release of a personal hobby project, I don't know. I trialed the tool idea with the Reddit community, one of the last sources of social media that provides brutally honest feedback from anonymous people worldwide. From that experience, I learned that there was interest. A gracious respondent to my post even downloaded and tested the tool, providing feedback for improvements. The community interest sparked my motivation to begin iterating through minor releases, patching, and improving the tool daily to get it to a reliable state. The first major technical change I implemented was refactoring the manual parsing process of Terraform plan files. Initially, I wanted to understand how Terraform structured its plan files, unmarshal the JSON, and generate the Mermaid diagram. I decided to look back at Rover's internals for inspiration and discovered the terraform-exec Go module, which allowed me to harness the power of terraform graph functionality to achieve my goal. With this new knowledge, I refactored Terramaid to use Cobra for a better CLI experience, harnessed graph functionality and gographviz to convert the DOT output into Mermaid, and added documentation, demos, and example GitHub Actions and GitLab CI Pipeline examples. It was even more exciting when the community contributed improvements to the documentation, Dockerfiles, and added the tool to Homebrew's core repository. Being able to brew install terramaid has made me very happy. We have made many strides with Terramaid, and I've never been as excited as I am to sit down, receive feedback from the community, and develop a vision for the future of visualizations in the Terraform and Opentofu ecosystem.
Future Development
A week after the initial release, I was asked to demo the tool to a larger group in the cloud community. The feedback I received was exactly what I needed to begin mapping the next steps for development. There are a few major areas where Terramaid can improve to provide a better experience. The first is in the generation of the Mermaid diagram itself. Terraform's graph functionality provides a plethora of information, from labels to provider details, and visualizing this effectively can be challenging. My ultimate goal is to provide a customizable interface for engineers to harness Terramaid to create diagrams that work for them. This starts with perfecting flowcharts before branching out to other potentially useful forms like block diagrams or even mind maps. Once we optimize in this area, the possibilities for expansion are endless.
Another area to address is how we handle large amounts of resources, as not every resource and module should have its own dedicated block. Should we provide a truncated view to the engineer? Should we utilize Mermaid's syntax to create a better view of how many of each resource will be deployed? I tend to lean toward the latter, but I am open to suggestions and recommendations from the community.
Additionally, we need to consider how we treat modules. Will they always be a black box, or is there a better way to dive deeper and visualize which resources will be provisioned as a result? This is an area that will require further investigation, but I am optimistic.