By Wes Dean
Introduction
MegaLinter is an Open Source tool powered and sponsored by the fine folks at OX Security. Getting started with MegaLinter can be a little daunting, so here are 4 steps to getting started with MegaLinter. Flexion uses MegaLinter to tidy our code, verify best practices, and keep on the lookout for potential security vulnerabilities.
1. Get the source code into a repository
The first step is to get your project’s source code into a repository. This can be a hosted service like GitHub or GitLab, or it could be an Open Source tool like Gitea. Regardless of which system you use, how branches are configured, and how merges are handled, the key thing is to get your project’s source code into a repository. Individual developers on the project should have access to the repository and they should be pushing changes to the repository regularly.
2. Configure the linters you’ll be using
Next, set up the linters and scanners you’ll be using. Linters are tools like eslint, flake8, pmd, etc. that look at the source code and find patterns that deviate from best practices. Many tools work just fine without additional configuration.
Sometimes your team has adopted stylistic conventions that are different from what the linters are expecting. For example, suppose your team uses one space between content and comments in a YAML file (that is, the way prettier does it); yamllint, in its default configuration, expects two or more spaces between content and comments. (link to docs). So, right from the start, prettier will reformat YAML one way and yamllint will complain about it. That’s not very helpful.
Scanners are like linters, but they typically look for different kinds of problems like secrets, credentials, passwords, tokens, API keys, certificates, etc.. Some scanners (e.g., gitleaks) look for “high entropy strings” which are best not included with source code. However, there may be legitimate reasons for certain strings, such as pinned hashes for specific dependency releases. Having one scanner telling you to pin dependencies and another scanner telling you that including dependency pins may be problematic is similarly unhelpful. Situations like these are frustrating – it’s like trying to convince lovers of two different soft drinks to agree on which one is better. Good luck with that.
The cool thing about situations like these is that they’re not specific to MegaLinter. We’re just talking about setting up tools that may run in any context, such as an Integrated Development Environment (IDE), pre-commit hooks, etc. Even if MegaLinter is never included, having tool configurations all set helps make sure tools are used the same way across all of the developers’ environments.
3. Configure MegaLinter
Now things get even more interesting. MegaLinter is endlessly configurable. All of the linters and scanners MegaLinter includes can be configured with command line options, environment variables, flags, switches, knobs, and levers. Linters can be enabled or disabled, errors can be suppressed, pre- and post-commands can be run, and much, much more.
MegaLinter is a great tool and it really helps our projects run smoother. That said, there are times when it’s less than helpful to spell-check source code. If you’re like me and you don’t need MegaLinter to complain that “JavaScript” isn’t a word (spoiler alert: it is), there’s a fix for that. Disable the spell checker. If they’re not helpful, turn the problematic linters off! If you want to test MegaLinter configuration, MegaLinter can be run locally (e.g., via MegaLinter Docker).
Also, MegaLinter will create a directory (folder) named “megalinter-reports” where log files will be stored. It’s usually not very helpful to have the logs added to a GitHub repository, so an entry in the repository’s .gitignore file may be a good idea, especially in situations where files are being generated or updated and pushed back into the repository.
4. Make MegaLinter run automatically
This can be the most intimidating step in the process. Depending on what Continuous Integration / Continuous Delivery (or Deployment) system the project is using, the individual steps may vary. Don’t worry! MegaLinter’s got you covered!! Using GitHub? Done. Using GitLab? Gotcha. Jenkins? Piece of cake. In fact, more often than not, MegaLinter has a configuration file that can be dropped right into place, often with little or no modification. For example, using MegaLinter as a GitHub Action involves adding a single file to the repository. It’s literally, “add and commit this file to the repository and push the change” and that’s it. By default, when MegaLinter runs, if it finds anything it recommends changing, MegaLinter will fail the build. Sometimes this is what you want. Sometimes it’s an incredible nuisance to have a quality gate check fail because there was an extra space in some file somewhere.
This can be especially problematic when adding MegaLinter to an established project where code wasn’t previously linted. Have no fear, MegaLinter’s got you covered. “Errors” (conditions that would warrant the build failing) can be disabled so that the messages are shown, but the build still succeeds. This can be done on a per-linter basis (e.g., yamllint warnings don’t fail the build but security scanners will) or for the entire repository. Check out the DISABLE_ERRORS configuration variable in the documentation.
Bonus: Tie MegaLinter into other systems
Congratulations! Your project is on its way to ever-greater levels of great greatness!! The fun doesn’t end there. MegaLinter can be integrated into other systems, too. For example, MegaLinter can produce SARIF files that can be ingested into GitHub, DefectDojo, and more. Logs and build artifacts can be stored for future reference. GitHub, GitLab, BitBucket, and Azure PRs can be decorated with scan results. For many file types, MegaLinter can apply fixes (e.g., by running prettier) and push the updated files back into the repository. MegaLinter has this documented well; setting it up is a matter of providing an access token (e.g., a GitHub PAT) and setting a few flags.
Conclusion
Now that you’ve learned the basics of getting started with MegaLinter, you’re well on your way to improving your code quality. But there’s more to explore! In the next part of this series, we’ll dive deeper into a variety of tips, tricks, and customization options to help you maximize the power of MegaLinter.
Have questions? Contact us to learn how team Flexion can help streamline your development process.
Wes Dean, a Senior DevSecOps Engineer at Flexion, brings his extensive experience in the UNIX and Linux world since the early 1990s to his role. He supports a variety of U.S. Federal agencies, helping them work safer, faster, more efficiently, and more securely. Wes’s unique position as a member of the CMS Open Source Program Office Advisory Board’s CMS Source Code Stewardship Taskforce underscores his expertise and credibility. He is also a staunch supporter of MegaLinter and a contributor to the tool’s prose scanning functionality, among other improvements.
