Refactoring the load system

Description


The existing load system was quite complex, which made it hard to change. And it needed to change, in order to support Unified reference detection

This Initiative is about biting the bullet to break the existing load system into smaller parts that aren’t as difficult to change.


Status: In progress

The refactor itself has been implemented! :tada: Switch to LoadContext based loading :tada:

But to call it completed, we should clean up old code.

Champion

Avatar Beanow @Beanow


Benefits


  • Divides and conquers the complexity of the load system.
  • More robust unit tests.
  • Easier to implement changes in the future.
    (better tests = more confidence, less complexity = less headaches for devs)
  • Many implicit features of the load system become explicit.
    For example, we now have a definition of the “SourceCred data directory” we didn’t have before.

Implementation Plan


Apply head to keyboard


Deliverables


A design document which outlines the new concepts we’ll use to break the load system into smaller parts. And plenty of PRs to implement the refactor.


Dependencies


None at the moment.


References


Discussion / trackers

Related refactors, used for reference


Contributions


Design doc

LoadPlan version (concept rejected)

LoadContext version (concept accepted)

Prerequisite refactoring

Cleanup :broom:


1 Like

@decentralion can you give this a look to see if it makes sense? I’ve included some of your related work. Though not all, as WeightedGraph could be it’s own Initiative / Artifact.

Some I’m directly using, so they’re contributions:

Some are replaced but provided input. I’ve opted for references here, because they were intended to support WeightedGraph, not directly to address the complexity of load:

Fun fact, if you’re wondering why so many contributions. Some of the first work started way back in mid-December. It’s been a loooong time coming :sweat_smile:

image

Looks good to me.

Wow. So much work!

Selfish question: will this allow me to run SourceCred on /Bitcoin and other old, messy GitHub repos with lots of dirty data that SourceCred currently chokes on?

No, although @wchargin is working on a change that likely will. See: GraphQL Mirror fidelity awareness

Oooooh. :heart:'d Initiative for what it’s worth.