dogfooding our plugin system
Let me propose a similar but countervailing suggestion. Pull plugins out of core, at an interface level and a filesystem directory level, but keep them in the main repository for a while yet. Collocate any new plugins alongside the existing ones in the main repository. If this is all easy to do, excellent! We can migrate them all out of the repository without difficulty. If not—if we find that new plugins or changes to our current plugins are suggesting changes to the core APIs—then it will be a good thing that we held off on physically splitting them out. It will be good that we added the coupling that we did, because the alternative would have been adding backdoor “experimental only!” public APIs, doing a version bump, updating dependencies in the subproject, using the new hacky API, adding a half-hearted TODO to migrate off of it eventually, and more likely than not ending up with a long-standing blemish on the final product.
Our larger danger here is not that we might fail to extract enough. It is that we might settle onto the wrong abstraction. A plugin system is one of the heaviest-weight kinds of abstractions that you can get (second only, maybe, to code generation systems). This makes it especially hard to revise design decisions. The multi-stage “A/AB/B” compatibility windows and other overhead that multirepos enforce are great once you’re at the point that you care about compatibility. When you’re still exploring, they’re severely limiting.
Will Tracz popularized a “rule of threes”: write three implementations of an interface before solidifying it. If you write just one, you’re probably missing important points in the feature space. If you write two, you’ll have a better idea, but it may be difficult or awkward to adopt new implementations. If you write three, you’ll be fine. Where do we stand today? We have the GitHub and Discourse plugins, and also the identity plugin. The latter is quite different from the other two, in ways that reveal that we really don’t understand all the pieces yet.
As proof of feasibility: the src/graphql/ subtree has been carefully developed as an independent package. It has no dependencies whatsoever on the rest of SourceCred (core or plugins), and never has. Being in the same repository didn’t make this any harder. We just treated the package boundary as if it were a repository boundary, even though there was no such technical restriction. And the monorepo property has been helpful, as we’ve been able to make changes like PR #961 and PR #1337 atomically, to say nothing of developing the upcoming fidelity enhancements!